Section 5.3
Operators on Hilbert spaces
We are now going to discuss linear maps between Hilbert spaces. In most contexts, Hilbert spaces are spaces of functions. So a linear map between Hilbert spaces is usually called a linear operator instead of a linear map. The map $f \mapsto \frac{df}{dx}$ being called a differential operator explains the terminology.
Given an operator between Hilbert spaces, we want to define how “large” that operator is. More precisely, this measures how large $Tv$ can be compared to $v$.
Let $V, W$ be finite-dimensional vector spaces over $k$, where $k = \mathbb{R}$ or $k = \mathbb{C}$. Assume that $\dim V \ge 1$. For a linear operator $T : V \to W$, we define its norm as
$$ \lVert T \rVert = \max_{\lVert v \rVert_V \le 1} \lVert Tv \rVert_W = \max_{v \neq 0} \frac{\lVert Tv \rVert_W}{\lVert v \rVert_V}. $$
(Here, $\lVert Tv \rVert_W$ is the norm of $Tv$ with respect to the inner product in $W$, and $\lVert v \rVert_V$ is the norm of $v$ with respect to the inner product in $V$.)
But before we start using this definition, we need to ask if there really is a maximum. Why cannot $\lVert Tv \rVert_W$ be indefinitely large with $\lVert v \rVert_V \le 1$? What if $\lVert Tv \rVert_W$ can take every value smaller than $1$ but never $1$, for instance? This is where we need the Heine–Borel theorem (Theorem 5.1.8). The unit ball
$$ B = \lbrace v \in V : \lVert v \rVert_V \le 1 \rbrace $$
is clearly bounded. To see that $B$ is closed, we use the fact that the norm is continuous. (See Exercise 5.2.E.) If $b_1, b_2, \ldots \in B$ and $\lim_{k \to \infty} b_k = b$ then $\lVert b \rVert_V = \lim_{k \to \infty} \lVert b_k \rVert_V \le 1$ because $\lvert b_k \rvert_V \le 1$ for all $k$. Therefore $B$ is bounded and closed, and Heine–Borel implies that our definition makes sense. This means that $\lVert Tv \rVert_W \le \lVert T \rVert \lVert v \rVert_V$ for all $v \in V$, and there exists a $v \neq 0$ such that $\lVert Tv \rVert_W = \lVert T \rVert \lVert v \rVert_V$.
Consider the Hilbert space $\mathbb{R}^2$, with the standard inner product. Compute the norm of the linear operator
$$ \begin{bmatrix} 1 & 2 \\ 3 & 4 \end{bmatrix}. $$
Let $V, W, U$ be finite-dimensional Hilbert spaces, and let $T : V \to W$ and $S : W \to U$ be linear operators. Show that
$$ \lVert ST \rVert \le \lVert S \rVert \lVert T \rVert, $$
and find examples of $S$ and $T$ such that $\lVert ST \rVert < \lVert S \rVert \lVert T \rVert$.
Let $V, W$ be finite-dimensional Hilbert spaces, and let $T, S : V \to W$ be linear operators. Show that
$$ \lVert T+S \rVert \le \lVert T \rVert + \lVert S \rVert. $$
Let $V, W$ be finite-dimensional Hilbert spaces, and fix $T : W \to V$ a linear operator. For each vector $v \in V$, there is a linear functional
$$ \langle v, T(-) \rangle_V : W \to k; \quad w \mapsto \langle v, Tw \rangle_V. $$
By the Riesz representation theorem (Theorem 5.2.8), this linear functional is uniquely representable.
Let $V, W$ be finite-dimensional Hilbert spaces, and let $T : W \to V$ be a linear operator. The adjoint operator $T^\dagger$ of $T$ is defined to satisfy
$$ \langle v, Tw \rangle_V = \langle T^\dagger v, w \rangle_W $$
for all $v \in V$ and $w \in W$.
Show that the adjoint operator $T^\dagger : V \to W$ is linear. (The above definition is at the level of sets.)
In the above setting, show that $(T^\dagger)^\dagger = T$.
Let $V, W, U$ be finite-dimensional Hilbert spaces, and let $T : V \to W$ and $S : W \to U$ be linear operators. Show that
$$ (ST)^\dagger = T^\dagger S^\dagger. $$
Take $V = \mathbb{C}^n$ and $W = \mathbb{C}^m$ (with standard inner products), and write $T : W \to V$ as an $n \times m$ matrix $T = (t_{ij})$ with complex entries. Its adjoint will be a linear map $T^\dagger : V \to W$, which will be an $m \times n$ matrix. What matrix is this going to be? The equation $\langle v, Tw \rangle_V = \langle T^\dagger v, w \rangle_W$ can be written as
$$ \sum_{i=1}^{n} \bar{v}_i \sum_{j=1}^{m} t_{ij} w_j = \sum_{j=1}^{m} \overline{(T^\dagger v)}_j w_j, $$
and this shows that $(T^\dagger v)_j = \sum_{i=1}^{n} \bar{t}_{ij} v_i$. That is, $T^\dagger$ is the conjugate transpose of $T$ defined so that the $i$th column $j$th row entry of $T^\dagger$ is the complex conjugate of the $j$th column $i$th row entry of $T$. We can also write $T^\dagger = \bar{T}^\ast$ if we wish, where $\ast$ means transpose and $\overline{\;\;}$ means taking complex conjugate of all the entries.
Let $V, W$ be finite-dimensional Hilbert spaces over $k$. We can define an inner product structure on $V \oplus W$ by
$$ \langle (v_1, w_1), (v_2, w_2) \rangle = \langle v_1, v_2 \rangle + \langle w_1, w_2 \rangle, $$
and $V \oplus W$ is a finite-dimensional Hilbert space with this inner product. Let $T : W \to V$ be a linear operator, and define subspaces $X, Y \subseteq V \oplus W$ as
$$ X = \lbrace (Tw, w) : w \in W \rbrace , \quad Y = \lbrace (v, -T^\dagger v) : v \in V \rbrace . $$
Show that $X^\perp = Y$ in $V \oplus W$.
Let $V$ be a finite-dimensional Hilbert space and let $T : V \to V$ be a linear operator. Show that $T$ is unitary (or orthogonal) if and only if $T^{-1} = T^\dagger$.
So a square matrix $Q$ with real entries is orthogonal if and only if $Q Q^\ast = Q^\ast Q = I$ and a square matrix $U$ with complex entries is unitary if and only if $U U^\dagger = U^\dagger U = I$.
We now consider the case when $V = W$. In this case, we can compare the two linear operators $T, T^\dagger : V \to V$.
Let $V$ be a finite-dimensional Hilbert space over $k$ and let $T : V \to V$ be a linear operator. We say that $T$ is self-adjoint if $T = T^\dagger$. When $k = \mathbb{R}$, a synonym for self-adjoint is symmetric and when $k = \mathbb{C}$, a self-adjoint operator is also called a Hermitian operator.
The condition $T = T^\dagger$ just means
$$ \langle Tv, w \rangle = \langle v, Tw \rangle $$
for all $v, w \in V$. If $V = \mathbb{C}^n$, the $n \times n$ matrix $T$ is Hermitian if and only if its conjugate transpose is itself.
Why are we interested in self-adjoint operators? The main reason is that the spectral theorem, which we will learn in the next section, applies to self-adjoint operators. Let us get a glimpse of the theorem.
Let $V$ be a finite-dimensional Hilbert space, and let $T : V \to V$ be a self-adjoint operator. Show that every eigenvalue of $T$ is real, i.e., if $Tv = \lambda v$ for $v \neq 0$ and $\lambda \in k$ then $\lambda \in \mathbb{R}$. (This statement is nontrivial only when $k = \mathbb{C}$.)
Let $V$ be a finite-dimensional Hilbert space, and let $T : V \to V$ be a self-adjoint operator. If $v_1$ is an eigenvector with eigenvalue $\lambda_1$, $v_2$ is an eigenvector with eigenvalue $\lambda_2$, and $\lambda_1 \neq \lambda_2$, show that $v_1$ and $v_2$ are orthogonal, i.e., $\langle v_1, v_2 \rangle = 0$.
This is what it means. When $V$ is a finite-dimensional Hilbert space with $T$ an operator on it, for each $\lambda \in \mathbb{R}$ we can define
$$ V_\lambda = \ker(T - \lambda I) = \lbrace v \in V : Tv = \lambda v\rbrace . $$
Then $V_\lambda$ is nonzero only for finitely many $\lambda$, and all these vector spaces are orthogonal to each other.
Let $V$ be a finite-dimensional Hilbert space with $T$ an operator on it. For a vector $v \in V$, show that there exists at most one representation
$$ v = v_1 + v_2 + \cdots + v_n $$
with each $v_i \neq 0$ an eigenvector with eigenvalue $\lambda_i$, and $\lambda_1 < \lambda_2 < \cdots < \lambda_n$.