Operators on Inner Product Spaces

7A. Self-Adjoint and Normal Operators

1) Adjoint

Adjoint

For

$$T\in\mathcal{L}(V,W),$$

the adjoint of $T$ is the linear map

$$T^*:W\to V$$

such that

$$\langle Tv,w\rangle = \langle v,T^*w\rangle$$

for every $v\in V$ and $w\in W$.

Meaning: The adjoint moves a linear map from one side of an inner product to the other.

2) Properties of the Adjoint

Addition

$$(S+T)^* = S^*+T^*.$$

Meaning: The adjoint preserves addition.

Scalar Multiplication

$$(\lambda T)^* = \overline{\lambda}T^*.$$

Meaning: Taking the adjoint conjugates scalar factors.

Double Adjoint

$$(T^*)^* = T.$$

Meaning: Taking the adjoint twice returns the original map.

Product

$$(ST)^* = T^*S^*.$$

Meaning: Taking the adjoint reverses the order of composition.

Identity

$$I^*=I.$$

Meaning: The identity operator is its own adjoint.

Inverse

If $T$ is invertible, then

$$(T^*)^{-1} = (T^{-1})^*.$$

Meaning: Taking an adjoint is compatible with taking an inverse.

3) Null Space and Range of the Adjoint

Null Space of the Adjoint

$$\operatorname{null}T^* = (\operatorname{range}T)^\perp.$$

Meaning: $T^*$ sends exactly the vectors orthogonal to the range of $T$ to zero.

Range of the Adjoint

$$\operatorname{range}T^* = (\operatorname{null}T)^\perp.$$

Meaning: The range of $T^*$ consists of the directions orthogonal to the null space of $T$.

4) Conjugate Transpose

Conjugate Transpose

For a matrix $A$, the conjugate transpose $A^*$ is obtained by transposing $A$ and conjugating every entry.

$$(A^*)_{j,k} = \overline{A_{k,j}}.$$

Meaning: $A^*$ is the matrix counterpart of the adjoint.

Real Matrix

If $A$ has only real entries, then

$$A^*=A^t.$$

Meaning: For real matrices, conjugate transpose is just transpose.

5) Matrix of the Adjoint

Adjoint Matrix

With respect to orthonormal bases,

$$\mathcal{M}(T^*) = \mathcal{M}(T)^*.$$

Meaning: In orthonormal coordinates, taking an adjoint corresponds exactly to taking a conjugate transpose.

6) Self-Adjoint Operator

Self-Adjoint

An operator $T\in\mathcal{L}(V)$ is self-adjoint if

$$T=T^*.$$

Meaning: A self-adjoint operator is the operator analogue of a real number.

Inner Product Criterion

$$\langle Tv,w\rangle = \langle v,Tw\rangle$$

for all $v,w\in V$.

Meaning: A self-adjoint operator can be moved across an inner product without changing it.

7) Eigenvalues of Self-Adjoint Operators

Real Eigenvalues

Every eigenvalue of a self-adjoint operator is real.

Meaning: Self-adjoint operators cannot have nonreal eigenvalues.

8) Normal Operator

Normal

An operator $T\in\mathcal{L}(V)$ is normal if

$$TT^* = T^*T.$$

Meaning: A normal operator commutes with its adjoint.

Self-Adjoint Implies Normal

Every self-adjoint operator is normal.

Meaning: Self-adjoint operators form an important subclass of normal operators.

9) Norm Criterion for Normality

Normality Criterion

$$T\text{ is normal} \iff \|Tv\|=\|T^*v\|$$

for every $v\in V$.

Meaning: A normal operator and its adjoint stretch every vector by the same amount.

10) Null Space and Range of a Normal Operator

Null Spaces

$$\operatorname{null}T = \operatorname{null}T^*.$$

Meaning: A normal operator and its adjoint kill exactly the same vectors.

Ranges

$$\operatorname{range}T = \operatorname{range}T^*.$$

Meaning: A normal operator and its adjoint reach the same subspace.

Orthogonal Decomposition

$$V = \operatorname{null}T \oplus \operatorname{range}T.$$

Meaning: For a normal operator, the null space and range split the whole space orthogonally.

11) Eigenvectors of a Normal Operator

Adjoint Eigenvector

If

$$Tv=\lambda v,$$

then

$$T^*v = \overline{\lambda}v.$$

Meaning: A normal operator and its adjoint have the same eigenvectors, with conjugate eigenvalues.

Orthogonal Eigenvectors

Eigenvectors corresponding to distinct eigenvalues of a normal operator are orthogonal.

Meaning: Different eigenvalues of a normal operator give perpendicular eigendirections.

7B. Spectral Theorem

1) Real Spectral Theorem

Real Spectral Theorem

Suppose $\mathbb{F}=\mathbb{R}$.

Then

$$T\text{ is self-adjoint}$$

if and only if $V$ has an orthonormal basis consisting of eigenvectors of $T$.

Equivalently, $T$ has a diagonal matrix with respect to some orthonormal basis.

Meaning: Every real self-adjoint operator can be orthogonally diagonalized.

2) Complex Spectral Theorem

Complex Spectral Theorem

Suppose $\mathbb{F}=\mathbb{C}$.

Then

$$T\text{ is normal}$$

if and only if $V$ has an orthonormal basis consisting of eigenvectors of $T$.

Equivalently, $T$ has a diagonal matrix with respect to some orthonormal basis.

Meaning: Every complex normal operator can be unitarily diagonalized.

3) Spectral Theorem Summary

Real Case

$$T=T^* \iff T\text{ is orthogonally diagonalizable}.$$

Meaning: Self-adjointness is exactly the condition for orthonormal diagonalization over $\mathbb{R}$.

Complex Case

$$TT^*=T^*T \iff T\text{ is unitarily diagonalizable}.$$

Meaning: Normality is exactly the condition for orthonormal diagonalization over $\mathbb{C}$.

7C. Positive Operators

1) Positive Operator

Positive Operator

An operator $T\in\mathcal{L}(V)$ is positive if $T$ is self-adjoint and

$$\langle Tv,v\rangle \ge 0$$

for every $v\in V$.

Meaning: A positive operator has a nonnegative quadratic value in every direction.

2) Characterization of Positive Operators

Eigenvalue Criterion

A self-adjoint operator $T$ is positive if and only if all its eigenvalues are nonnegative.

Meaning: Positivity can be checked directly from the spectrum.

Diagonal Criterion

$T$ is positive if and only if it has a diagonal matrix with nonnegative diagonal entries with respect to some orthonormal basis.

Meaning: A positive operator acts like nonnegative scaling along orthogonal directions.

3) Square Root of an Operator

Square Root

An operator $R$ is a square root of $T$ if

$$R^2=T.$$

Meaning: Applying $R$ twice gives $T$.

4) Positive Square Root

Existence of Positive Square Root

Every positive operator $T$ has a positive square root.

Meaning: Positive operators behave like nonnegative real numbers with respect to square roots.

Uniqueness

Every positive operator has exactly one positive square root.

It is denoted by

$$\sqrt{T}.$$

Meaning: There is one uniquely determined positive operator whose square is $T$.

5) Equivalent Characterizations of Positivity

Positive Operator Characterization

The following are equivalent:

$$T\text{ is positive},$$ $$T\text{ is self-adjoint with nonnegative eigenvalues},$$ $$T=R^2 \text{ for some positive }R,$$

and

$$T=R^*R \text{ for some }R\in\mathcal{L}(V).$$

Meaning: Positivity can be recognized through eigenvalues, square roots, or a factorization of the form $R^*R$.

7D. Isometries, Unitary Operators, and Matrix Factorization

1) Isometry

Isometry

A linear map

$$S\in\mathcal{L}(V,W)$$

is an isometry if

$$\|Sv\| = \|v\|$$

for every $v\in V$.

Meaning: An isometry preserves vector lengths.

2) Characterization of Isometries

Adjoint Criterion

$$S\text{ is an isometry} \iff S^*S=I.$$

Meaning: An isometry does not change lengths because its adjoint reverses its action on the domain.

Inner Product Preservation

$$\langle Su,Sv\rangle = \langle u,v\rangle.$$

Meaning: Isometries preserve both lengths and angles.

Orthonormal Basis Criterion

An isometry maps an orthonormal basis of $V$ to an orthonormal list in $W$.

Meaning: Isometries preserve orthonormal structure.

3) Unitary Operator

Unitary Operator

An operator $S\in\mathcal{L}(V)$ is unitary if it is an invertible isometry.

Meaning: A unitary operator preserves all inner-product geometry and can be reversed.

4) Characterization of Unitary Operators

Adjoint and Inverse

$$S^*S = SS^* = I.$$

Thus,

$$S^{-1} = S^*.$$

Meaning: The inverse of a unitary operator is its adjoint.

Orthonormal Basis Preservation

A unitary operator maps every orthonormal basis to another orthonormal basis.

Meaning: Unitary operators change coordinates without changing geometric structure.

5) Eigenvalues of Unitary Operators

Absolute Value of Eigenvalues

Every eigenvalue $\lambda$ of a unitary operator satisfies

$$|\lambda|=1.$$

Meaning: Unitary operators do not stretch eigenvectors; they only rotate or reflect them.

6) Complex Unitary Operators

Spectral Description

For a complex inner product space,

$$S\text{ is unitary}$$

if and only if $V$ has an orthonormal basis of eigenvectors whose eigenvalues all satisfy

$$|\lambda|=1.$$

Meaning: A complex unitary operator is diagonalizable using unit-length complex scalings.

7) Unitary Matrix

Unitary Matrix

A square matrix $Q$ is unitary if its columns form an orthonormal basis.

Equivalently,

$$Q^*Q = QQ^* = I.$$

Meaning: A unitary matrix preserves Euclidean lengths and inner products.

8) QR Factorization

QR Factorization

If a square matrix $A$ has linearly independent columns, then

$$A=QR,$$

where $Q$ is unitary and $R$ is upper triangular with positive diagonal entries.

Meaning: QR factorization separates a matrix into an orthonormal part and an upper-triangular part.

Uniqueness

With positive diagonal entries in $R$, the factorization

$$A=QR$$

is unique.

Meaning: The positivity condition removes ambiguity in the QR decomposition.

9) Positive Definite Matrix

Positive Definite

A square matrix $B$ is positive definite if

$$B^*=B$$

and

$$\langle Bx,x\rangle>0$$

for every nonzero $x$.

Meaning: A positive definite matrix is the matrix form of an invertible positive operator.

10) Cholesky Factorization

Cholesky Factorization

If $B$ is positive definite, then there exists a unique upper-triangular matrix $R$ with positive diagonal entries such that

$$B=R^*R.$$

Meaning: A positive definite matrix can be factored into a triangular matrix and its conjugate transpose.

7E. Singular Value Decomposition

1) Properties of $T^*T$

Positive Operator

For

$$T\in\mathcal{L}(V,W),$$

the operator

$$T^*T$$

is positive.

Meaning: $T^*T$ always has nonnegative eigenvalues and an orthonormal eigenbasis.

Null Space

$$\operatorname{null}(T^*T) = \operatorname{null}T.$$

Meaning: $T^*T$ loses exactly the same input directions as $T$.

Rank

$$\dim\operatorname{range}T = \dim\operatorname{range}(T^*T).$$

Meaning: $T$ and $T^*T$ contain the same amount of nonzero directional information.

2) Singular Values

Singular Values

The singular values of $T$ are the nonnegative square roots of the eigenvalues of

$$T^*T.$$

They are listed in decreasing order.

Meaning: Singular values measure how strongly $T$ stretches orthogonal input directions.

3) Zero Singular Value

Injectivity Criterion

$$T\text{ is injective} \iff 0\text{ is not a singular value of }T.$$

Meaning: A zero singular value indicates that some nonzero direction is completely collapsed.

4) Positive Singular Values and Rank

Rank from Singular Values

The number of positive singular values of $T$ equals

$$\dim\operatorname{range}T.$$

Meaning: The number of nonzero singular values is the rank of the linear map.

5) Isometry and Singular Values

Isometry Criterion

$$S\text{ is an isometry} \iff \text{all singular values of }S\text{ equal }1.$$

Meaning: An isometry stretches every independent direction by exactly one.

6) Singular Value Decomposition

Singular Value Decomposition

Let

$$s_1,\ldots,s_m$$

be the positive singular values of $T$.

Then there exist orthonormal lists

$$e_1,\ldots,e_m$$

in $V$ and

$$f_1,\ldots,f_m$$

in $W$ such that

$$Tv = \sum_{k=1}^{m} s_k\langle v,e_k\rangle f_k.$$

Meaning: Every linear map can be decomposed into orthogonal input directions, scalar stretch factors, and orthogonal output directions.

7) Singular Vectors

Right Singular Vectors

The vectors $e_k$ satisfy

$$T^*T e_k = s_k^2e_k.$$

Meaning: Right singular vectors are eigenvectors of $T^*T$.

Left Singular Vectors

For $s_k>0$,

$$f_k = \frac{Te_k}{s_k}.$$

Meaning: Left singular vectors are the normalized output directions corresponding to the right singular vectors.

8) Diagonal Form of a Linear Map

SVD Diagonal Form

With suitable orthonormal bases of $V$ and $W$, the matrix of $T$ has only singular values on its diagonal and zeros elsewhere.

Meaning: Unlike eigenvalue diagonalization, SVD gives a diagonal form for every linear map.

9) Matrix Version of SVD

Matrix SVD

If an $M$-by-$n$ matrix $A$ has rank $m$, then

$$A=BDC^*,$$

where

• $B$ has orthonormal columns,
• $D$ is diagonal with positive singular values,
• $C$ has orthonormal columns.

Meaning: Any matrix can be separated into orthogonal input directions, singular-value scaling, and orthogonal output directions.

10) Adjoint from the SVD

Adjoint Formula

If

$$Tv = \sum_{k=1}^{m} s_k\langle v,e_k\rangle f_k,$$

then

$$T^*w = \sum_{k=1}^{m} s_k\langle w,f_k\rangle e_k.$$

Meaning: The adjoint reverses the input and output singular directions while keeping the same singular values.

11) Pseudoinverse from the SVD

Pseudoinverse Formula

$$T^\dagger w = \sum_{k=1}^{m} \frac{\langle w,f_k\rangle}{s_k}e_k.$$

Meaning: The pseudoinverse reverses each nonzero singular direction by replacing $s_k$ with $1/s_k$.

7F. Consequences of Singular Value Decomposition

1) Norm of a Linear Map

Operator Norm

For

$$T\in\mathcal{L}(V,W),$$

define

$$\|T\| = \max_{\|v\|\le1}\|Tv\|.$$

Meaning: The operator norm is the largest amount by which $T$ can stretch a unit vector.

2) Norm and Singular Values

Largest Singular Value

If $s_1$ is the largest singular value of $T$, then

$$\|T\|=s_1.$$

Meaning: The strongest stretching factor of $T$ is its operator norm.

3) Fundamental Operator Norm Inequality

Norm Bound

For every $v\in V$,

$$\|Tv\| \le \|T\|\|v\|.$$

Meaning: The operator norm gives a universal bound on how much $T$ can stretch any vector.

4) Basic Properties of the Operator Norm

Positivity

$$\|T\|\ge0.$$

Zero

$$\|T\|=0 \iff T=0.$$

Scalar Multiplication

$$\|\lambda T\| = |\lambda|\|T\|.$$

Triangle Inequality

$$\|S+T\| \le \|S\|+\|T\|.$$

Meaning: The operator norm behaves like an ordinary vector norm.

5) Norm of the Adjoint

Adjoint Norm

$$\|T^*\| = \|T\|.$$

Meaning: A linear map and its adjoint have the same maximum stretching factor.

6) Best Low-Rank Approximation

Best Rank-$k$ Approximation

Suppose

$$s_1\ge\cdots\ge s_m>0$$

are the positive singular values of $T$.

For $1\le k<m$,

$$\min_{\dim\operatorname{range}S\le k} \|T-S\| = s_{k+1}.$$

Meaning: The best rank-$k$ approximation is obtained by discarding all but the $k$ largest singular values.

Truncated SVD

If

$$T_kv = \sum_{j=1}^{k} s_j\langle v,e_j\rangle f_j,$$

then

$$\|T-T_k\| = s_{k+1}.$$

Meaning: Truncating the SVD gives the optimal lower-rank approximation.

7) Polar Decomposition

Polar Decomposition

For every operator

$$T\in\mathcal{L}(V),$$

there exists a unitary operator $S$ such that

$$T = S\sqrt{T^*T}.$$

Meaning: Every operator can be written as a unitary geometric transformation followed by positive stretching.

8) Positive Part of the Polar Decomposition

Positive Factor

The positive factor in the polar decomposition is

$$\sqrt{T^*T}.$$

Meaning: $\sqrt{T^*T}$ contains the pure stretching information of $T$.

9) Relation to Singular Values

Singular Values of the Positive Factor

The eigenvalues of

$$\sqrt{T^*T}$$

are the singular values of $T$.

Meaning: The positive factor in the polar decomposition stretches along singular directions by the singular values.