Section 5.5
Positivity of operators
We now restrict our attention further to a smaller class of operators.
Let $V$ be a finite-dimensional Hilbert space. An operator $T : V \to V$ is called
positive definite if it is self-adjoint and $\langle v, Tv \rangle > 0$ for all nonzero $v \in V$,
positive semi-definite if it is self-adjoint and $\langle v, Tv \rangle \ge 0$ for all $v \in V$.
Clearly, being positive definite is a stronger condition than being positive semi-definite, as the terminology suggests.
Let $V$ be a finite-dimensional Hilbert space. Show that the sesquilinear form
$$ V \times V \to k; \quad (x, y) \mapsto \langle x, Ty \rangle $$
defines an inner product on $V$ if and only if $T : V \to V$ is positive definite.
Let $V, W$ be finite-dimensional Hilbert spaces, and let $T : V \to W$ be a linear operator. Show that $T^\dagger T : V \to V$ is positive semi-definite.
Let $V$ be a finite-dimensional Hilbert space and $T : V \to V$ be a self-adjoint operator.
- (a)
Show that $T$ is positive definite if and only if all the eigenvalues of $T$ are positive.
- (b)
Show that $T$ is positive semi-definite if and only if all the eigenvalues of $T$ are nonnegative.
Let $V$ be a finite-dimensional Hilbert space, and $T : V \to V$ be a positive semi-definite operator. If $T$ is invertible, show that $T$ is positive definite. Conversely, show that a positive definite operator is necessarily invertible.
The good thing about positive definite or semi-definite operators is that there are many things we can do with positive numbers, e.g., taking the square root. Let $T : V \to V$ be a positive semi-definite operator. Apply the spectral theorem to get a decomposition
$$ T = U \Lambda U^{-1}. $$
The condition that $T$ is positive semi-definite is equivalent to all the diagonal entries of $\Lambda$ being nonnegative. Then we can define $\sqrt{\Lambda}$ as the diagonal matrix with diagonal entries the square root of the diagonal entries of $\Lambda$. Then it is clear that $(\sqrt{\Lambda})^2 = \Lambda$. Moreover, if we define the square root of $T$ as
$$ \sqrt{T} = U \sqrt{\Lambda} U^{-1}, $$
then $\sqrt{T}$ is positive semi-definite and $(\sqrt{T})^2 = T$. An uncomfortable fact about this construction is that the decomposition $T = U \Lambda U^{-1}$ is not unique. However, the following proposition tells us that the square root is well-defined.
Let $V$ be a finite-dimensional Hilbert space, and let $T : V \to V$ be a positive semi-definite operator. Then there exists a unique positive semi-definite operator $S : V \to V$ such that $S^2 = T$.
Existence of $S$ is demonstrated by the above construction. Let us now consider a positive semi-definite $S$ such that $S^2 = T$. Since both $S$ and $T$ are positive semi-definite, the spectral theorem gives decompositions
$$ \begin{aligned} V &= \bigoplus_{\lambda \ge 0} V_{\lambda}^{(S)}, \quad V_{\lambda}^{(S)} = \ker (S - \lambda I), \\ V &= \bigoplus_{\lambda \ge 0} V_{\lambda}^{(T)}, \quad V_{\lambda}^{(T)} = \ker (T - \lambda I). \end{aligned} $$
Then for $v \in V_\lambda^{(S)}$ we have $S^2 v = S (\lambda v) = \lambda^2 v$, and hence $V_{\lambda}^{(S)} \subseteq V_{\lambda^2}^{(T)}$. Since $V = \bigoplus_{\lambda \ge 0} V_{\lambda}^{(S)} = \bigoplus_{\lambda \ge 0} V_{\lambda^2}^{(T)}$, we have $V_{\lambda}^{(S)} = V_{\lambda^2}^{(T)}$ for all $\lambda \ge 0$. Therefore $S v = \sqrt{\lambda} v$ for $v \in V_\lambda^{(T)} = V_{\sqrt{\lambda}}^{(S)}$. This uniquely determines $S$.
Obviously, if we do not require $S$ to be positive semi-definite, there can be many self-adjoint $S$ such that $S^2 = T$. For instance, take $S = \pm I$ and $T = I$. In this case, we will have $V_{\lambda}^{(T)} = V_{\sqrt{\lambda}}^{(S)} \oplus V_{-\sqrt{\lambda}}^{(S)}$ in the above proof.
Let $V$ be a finite-dimensional Hilbert space and $T : V \to V$ be a self-adjoint operator (not necessarily positive definite). Show that there exists a unique self-adjoint operator $S : V \to V$ such that $S^3 = T$. Moreover, show that $S$ is positive semi-definite if $T$ is positive semi-definite.
We have noted that $T^\dagger T$ is positive semi-definite for every linear operator $T$. There is a decomposition based on this fact.
Let $V$ be a finite-dimensional Hilbert space, and let $T : V \to V$ be a linear operator. There exists a unitary operator $U : V \to V$ and a positive semi-definite operator $P : V \to V$ such that
$$ T = UP. $$
Moreover, if $T$ is invertible, such a decomposition is unique.
Note that if $T = UP$ then $T^\dagger T = (P U^{-1}) (UP) = P^2$. So the only choice we have is $P = \sqrt{T^\dagger T}$, which makes sense since $T^\dagger T$ is positive semi-definite. It now suffices to show that there is a unitary matrix $U$ such that
$$ T = U \sqrt{T^\dagger T}. $$
By Lemma 5.5.4, we need only check that $\lVert Tv \rVert = \lVert \sqrt{T^\dagger T} v \rVert$. But
$$ \lVert Tv \rVert^2 = \langle Tv, Tv \rangle = \langle v, T^\dagger T v \rangle = \langle v, (\sqrt{T^\dagger T})^2 v \rangle = \lVert \sqrt{T^\dagger T} v \rVert^2 $$
because $\sqrt{T^\dagger T}$ is self-adjoint.
If $T$ is invertible, $U$ is uniquely determined by the formula $U = T P^{-1}$.
Let $V, W$ be finite-dimensional Hilbert spaces, and let $T, S : V \to W$ be linear operators. If $\lVert Tv \rVert = \lVert Sv \rVert$ for all $v \in V$, then there exists an isometry $U : W \to W$ such that $S = UT$.
The condition implies that $Tv = 0$ if and only if $Sv = 0$, i.e., $\ker T = \ker S$. By the first isomorphism theorem, there exists a linear isomorphism $U_0 : \im T \to \im S$ such that $U_0 T = S$.
V maps to the image of T by T and to the image of S by S. A dashed isomorphism U zero between the two images makes the triangle commute.
The condition $\lVert Tv \rVert = \lVert Sv \rVert$ implies that $U_0$ is further an isometry.
Let $w_1, \ldots, w_n$ be an orthonormal basis of $W$ such that $w_1, \ldots, w_k$ is an orthonormal basis of $\im T \subseteq W$. Since $U_0$ is an isometry, the images $U_0 w_1, \ldots, U_0 w_k$ form an orthonormal basis of $\im S \subseteq W$. We now extend $U_0 w_1, \ldots, U_0 w_k$ to an orthonormal basis $U_0 w_1, \ldots, U_0 w_k, \tilde{w}_{k+1}, \ldots, \tilde{w}_n$ of $W$. Now we can extend $U_0 : \im T \to \im S$ to $U : W \to W$ by defining
$$ U w_i = \begin{cases} U_0 w_i & \text{if } 1 \le i \le k \\ \tilde{w}_{i} & \text{if } k+1 \le i \le n. \end{cases} $$
Because $U$ sends an orthonormal basis to an orthonormal basis, it is an isometry. Moreover, $S = UT$ by construction.
Find a polar decomposition for the matrix
$$ T = \begin{bmatrix} 3 & 4 & 4 & 3 \\ 1 & -2 & -2 & 1 \\ 3 & 2 & -2 & -3 \\ 1 & 0 & 0 & -1 \end{bmatrix}. $$
Let $V$ be a finite-dimensional Hilbert space, and let $T : V \to V$ be a linear operator. Show that there exist isometries $U : k^n \to V$ and $W : V \to k^n$ and an $n \times n$ diagonal matrix $\Lambda$ with nonnegative diagonal entries such that
$$ T = U \Lambda W. $$
Let $V$ be a finite-dimensional Hilbert space, and let $T : V \to V$ be a linear operator. Show that the eigenvalues of $T^\dagger T$ and of $T T^\dagger$ are identical.
The above decomposition $T = U \Lambda W$ is called singular value decomposition. But the interesting fact is that this decomposition holds for rectangular matrices as well.
Let $H, G$ be finite-dimensional Hilbert spaces with $n=\dim H$ and $m=\dim G$, and let $T : H \to G$ be a linear operator. Then there exist isometries $U : k^m \to G$ and $V : H \to k^n$, and a diagonal matrix with nonnegative diagonal entries $\Sigma : k^n \to k^m$ such that
$$ T = U \Sigma V. $$
(Here, a diagonal rectangular matrix is a matrix $\Sigma = (\sigma_{ij})$ such that $\sigma_{ij} = 0$ if $i \neq j$.)
Note that if we have a singular decomposition for $T^\dagger$, say $T^\dagger = U \Sigma V$, then we can take the adjoint of both sides and get
$$ T = V^\dagger \Sigma^\dagger U^\dagger. $$
Here, $V^\dagger$ and $U^\dagger$ are isometries as well, and hence we get a singular decomposition for $T$. Thus we may prove existence of a singular decomposition either for $T$ or $T^\dagger$, and hence we may as well assume that $n = \dim H \le \dim G = m$.
Because $\dim H \le \dim G$, there exists a linear subspace $G_0 \subseteq G$ such that $\im T \subseteq G_0$ and $\dim G_0 = \dim H$. Let us factor $T : H \to G$ as $T_0 : H \to G_0$ composed with $i : G_0 \hookrightarrow G$. Sending an orthonormal basis of $G_0$ to an orthonormal basis of $H$ defines an isometry $\Phi : G_0 \to H$.
H maps to G zero by T zero and then injects into G. An isometry Phi identifies G zero with H, with its inverse shown in the opposite direction.
We can now apply Exercise 5.5.G to the operator $\Phi T_0 : H \to H$. This gives a decomposition
$$ \Phi T_0 = U \Sigma V, \quad T_0 = (\Phi^{-1} U) \Sigma V. $$
But this $T_0$ is an operator $H \to G_0$, and to get the original $T$, we need to compose with the inclusion $i : G_0 \hookrightarrow G$. So
$$ T = i (\Phi^{-1} U) \Sigma V. $$
We need to take care of the embedding $i$. At this point, the operator $(\Phi^{-1} U) : k^n \to G_0$ is a composition of isometries, hence also isometry. Then we extend this isometry $\Phi^{-1} U$ to an isometry $\tilde{U} : k^m \to G$ so that the following diagram commutes.
k to the n maps by Sigma and an inclusion to k to the m, while Phi inverse U maps to G zero and then into G. A dashed isometry U tilde completes the diagram.
(To see existence of $\tilde{U}$, we can use Lemma 5.5.4. It is not hard to verify that the two operators $k^n \to k^m$ and $i \Phi^{-1} U$ satisfy the assumptions.) Now the composition $\tilde{\Sigma} : k^n \xrightarrow{\Sigma} k^n \hookrightarrow k^m$ is a diagonal matrix, and $\tilde{U}$ is a unitary operator so that $i \Phi^{-1} U \Sigma = \tilde{U} \tilde{\Sigma}$. Then
$$ T = \tilde{U} \tilde{\Sigma} V. $$
This finishes the proof.
Show that in the singular value decomposition of $T$, the diagonal entries of $\Sigma$ (including multiplicities) do not depend on the decomposition. The set of diagonal entries is called the singular values of $T$.
For instance, the singular values of a self-adjoint operator are the absolute values of its eigenvalues.
Find the singular value decomposition of the matrix
$$ T = \begin{bmatrix} 1 & 7 & -1 & -7 \\ 7 & -1 & -7 & 1 \\ 5 & 5 & 5 & 5 \end{bmatrix}. $$
Let $V, W$ be finite-dimensional Hilbert spaces, and let $T : V \to W$ be a linear operator. Let the singular values of $T$ be
$$ s_1 \ge s_2 \ge \cdots \ge s_n \ge 0, $$
where $n = \min(\dim V, \dim W)$. Show that
$$ s_k = \min\lbrace \lVert T - L \rVert : (L : V \to W) \text{ is a linear operator with } \rank L < k \rbrace , $$
for $1 \le k \le n$.