Eigenspaces

Eigenvalues and eigenvectors

If a linear transformation is represented by an \(n \times n\) matrix \(A\) and there exists a nonzero vector \(\mathbf{x} \in V\) such that \(A \mathbf{x} = \lambda \mathbf{x}\) for some \(\lambda \in \mathbb{K}\), then for this transformation \(\mathbf{x}\) is a natural choice to use as a basis vector for \(V\).

Definition 1: let \(A\) be a \(n \times n\) matrix, a scalar \(\lambda \in \mathbb{K}\) is defined as an eigenvalue of \(A\) if and only if there exists a vector \(\mathbf{x} \in V \backslash \{\mathbf{0}\}\) such that

\[ A \mathbf{x} = \lambda \mathbf{x}, \]

with \(\mathbf{x}\) defined as an eigenvector belonging to \(\lambda\).

This notion can be further generalized to a linear operator \(L: V \to V\) such that

\[ L(\mathbf{x}) = \lambda \mathbf{x}, \]

note that \(L(\mathbf{x}) = A \mathbf{x}\), which implies the similarity.

Furthermore it follows from the definition that any linear combination of eigenvectors is also a eigenvector of \(A\).

Theorem 1: let \(A\) be a \(n \times n\) matrix, a scalar \(\lambda \in \mathbb{K}\) is an eigenvalue of \(A\) if and only if

\[ \det (A - \lambda I) = 0. \]

Proof:

A scalar \(\lambda \in \mathbb{K}\) is an eigenvalue of \(A\) if and only if there exists a vector \(\mathbf{x} \in V \backslash \{\mathbf{0}\}\) such that

\[ A \mathbf{x} = \lambda \mathbf{x}, \]

obtains

\[ A \mathbf{x} - \lambda \mathbf{x} = (A - \lambda I) \mathbf{x} = \mathbf{0}, \]

which implies that \((A - \lambda I)\) is singular and \(\det(A - \lambda I) = 0\) by definition.

The eigenvalues \(\lambda\) may thus be determined from the characteristic polynomial of degree \(n\) that is obtained from \(\det (A - \lambda I) = 0\). In particular, the eigenvalues are the roots of this polynomial.

Theorem 2: let \(A\) be a \(n \times n\) matrix and let \(\lambda \in \mathbb{K}\) be an eigenvalue of \(A\). A vector \(\mathbf{x} \in V\) is an eigenvector of \(A\) corresponding to \(\lambda\) if and only if

\[ \mathbf{x} \in N(A - \lambda I) \backslash \{\mathbf{0}\}. \]

Proof:

Let \(A\) be a \(n \times n\) matrix, \(\mathbf{x} \in V\) is an eigenvector of \(A\) if and only if

\[ A \mathbf{x} = \lambda \mathbf{x}, \]

for an eigenvalue \(\lambda \in \mathbb{K}\). Therefore

\[ A \mathbf{x} - \lambda \mathbf{x} = (A - \lambda I) \mathbf{x} = \mathbf{0}, \]

which implies that \(\mathbf{x} \in N(A - \lambda I)\).

Which implies that the eigenvectors can be obtained by determining the corresponding null space of \(A - \lambda I\).

Definition 2: let \(L: V \to V\) be a linear operator and let \(\lambda \in \mathbb{K}\) be an eigenvalue of \(L\). Let the eigenspace \(E_\lambda\) of the corresponding eigenvalue \(\lambda\) be defined as

\[ E_\lambda = \{\mathbf{x} \in V \;|\; L(\mathbf{x}) = \lambda \mathbf{x}\} = N(A - \lambda I), \]

with \(L(\mathbf{x}) = A \mathbf{x}\).

It may be observed that \(E_\lambda\) is a subspace of \(V\) consisting of the zero vector and the eigenvectors of \(L\) or \(A.\)

Properties

Theorem 3: if \(\lambda_1, \dots, \lambda_n \in \mathbb{K}\) are distinct eigenvalues of an \(n \times n\) matrix \(A\) with corresponding eigenvectors \(\mathbf{x}_1, \dots \mathbf{x}_k \in V\backslash \{\mathbf{0}\}\), then \(\mathbf{x}_1, \dots \mathbf{x}_k\) are linearly independent.

Proof:

Will be added later.

If \(A \in \mathbb{R}^{n \times n}\) and \(A \mathbf{x} = \lambda \mathbf{x}\) for some \(\mathbf{x} \in V\) and \(\lambda \in \mathbb{K}\). Then

\[ A \mathbf{\bar x} = \overline{A \mathbf{x}} = \overline{\lambda \mathbf{x}} = \bar \lambda \mathbf{\bar x}. \]

The complex conjugate of an eigenvector of \(A\) is also an eigenvector of \(A\) with an eigenvalue \(\bar \lambda\).

Theorem 4: let \(A\) be a \(n \times n\) matrix and let \(\lambda_1, \dots, \lambda_n \in \mathbb{K}\) be the eigenvalues of \(A\). It follows that

\[ \det (A - \lambda I) = (\lambda_1 - \lambda)(\lambda_2 - \lambda) \cdots (\lambda_n - \lambda), \]

and

\[ \det (A) = \lambda_1 \lambda_2 \cdots \lambda_n. \]

Proof:

Let \(A\) be a \(n \times n\) matrix and let \(\lambda_1, \dots, \lambda_n \in \mathbb{K}\) be the eigenvalues of \(A\). It follows from the fundamental theorem of algebra that

\[ \det (A - \lambda I) = (\lambda_1 - \lambda)(\lambda_2 - \lambda) \cdots (\lambda_n - \lambda), \]

by taking \(\lambda = 0\) it follows that

\[ \det (A) = \lambda_1 \lambda_2 \cdots \lambda_n. \]

From \(\det (A) = \lambda_1 \lambda_2 \cdots \lambda_n\) it must follow that

\[ \mathrm{trace}(A) = \sum_{i=1}^n \lambda_i. \]

Theorem 5: let \(A\) and \(B\) be \(n \times n\) matrices. If \(B\) is similar to \(A\), then \(A\) and \(B\) have the same eigenvalues.

Proof:

Let \(A\) and \(B\) be similar \(n \times n\) matrices, then there exists a nonsingular matrix \(S\) such that

\[ B = S^{-1} A S. \]

Let \(\lambda \in \mathbb{K}\) be an eigenvalue of \(B\) then

\[ \begin{align*} 0 &= \det(B - \lambda I), \\ &= \det(S^{-1} A S - \lambda I), \\ &= \det(S^{-1}(A - \lambda I) S), \\ &= \det(S^{-1}) \det(A - \lambda I) \det(S), \\ &= \det(A - \lambda I). \end{align*} \]

Diagonalization

Definition 3: an \(n \times n\) matrix \(A\) is diagonalizable if there exists a nonsingular diagonalizing matrix \(X\) and a diagonal matrix \(D\) such that

\[ A X = X D. \]

We may now pose the following theorem.

Theorem 6: an \(n \times n\) matrix \(A\) is diagonalizable if and only if \(A\) has \(n \in \mathbb{N}\) linearly independent eigenvectors.

Proof:

Will be added later.

It follows from the proof that the column vectors of the diagonalizing matrix \(X\) are eigenvectors of \(A\) and the diagonal elements of \(D\) are the corresponding eigenvalues of \(A\). If \(A\) is diagonalizable, then

\[ A = X D X^{-1}, \]

it follows then that

\[ A^k = X D^k X^{-1}, \]

for \(k \in \mathbb{K}\).

Hermitian case

The following section is for the special case that a matrix is Hermitian.

Theorem 7: the eigenvalues of a Hermitian matrix are real.

Proof:

Let \(A\) be a Hermitian matrix and let \(\mathbf{x} \in V \backslash \{\mathbf{0}\}\) be an eigenvector of \(A\) with corresponding eigenvalue \(\lambda \in \mathbb{C}\). We have

\[ \begin{align*} \lambda \mathbf{x}^H \mathbf{x} &= \mathbf{x}^H (\lambda \mathbf{x}), \\ &= \mathbf{x}^H (A \mathbf{x}), \\ &= (\mathbf{x}^H A) \mathbf{x}, \\ &= (A^H \mathbf{x})^H \mathbf{x} , \\ &= (A \mathbf{x})^H \mathbf{x}, \\ &= (\lambda \mathbf{x})^H \mathbf{x}, \\ &= \bar \lambda \mathbf{x}^H \mathbf{x}, \end{align*} \]

since \(\bar \lambda = \lambda\) we must have that \(\lambda \in \mathbb{R}\).

Theorem 8: the eigenvectors of a Hermitian matrix corresponding to distinct eigenvalues are orthogonal.

Proof:

Let \(A\) be a Hermitian matrix and let \(\mathbf{x}_1, \mathbf{x}_2 \in V \backslash \{\mathbf{0}\}\) be two eigenvectors of \(A\) with corresponding eigenvalues \(\lambda_1, \lambda_2 \in \mathbb{C}[\lambda_1 \neq \lambda_2]\). We have

\[ \begin{align*} \lambda_1 \mathbf{x}_1^H \mathbf{x}_2 &= (\lambda_1 \mathbf{x}_1)^H \mathbf{x}_2, \\ &= (A \mathbf{x}_1)^H \mathbf{x}_2, \\ &= \mathbf{x}_1^H A^H \mathbf{x}_2, \\ &= \mathbf{x}_1^H A \mathbf{x}_2, \\ &= \mathbf{x}_1^H (\lambda_2 \mathbf{x}_2), \\ &= \lambda_2 \mathbf{x}_1^H \mathbf{x}_2, \end{align*} \]

since \(\lambda_1 \neq \lambda_2\) this must imply that \(\mathbf{x}_1^H \mathbf{x}_2 = 0\), implying orthogonality in terms of the Hermite scalar product.

Theorem 7 and 8 impose that the following definition can be used.

Definition 4: an \(n \times n\) matrix \(U\) is unitary if the column vectors of \(U\) form an orthonormal set in \(V\).

Thus, \(U\) is unitary if and only if \(U^H U = I\). Then it also follows that \(U^{-1} = U^H\). A real unitary matrix is an orthogonal matrix.

One may observe that theorem 8 implies that the diagonalizing matrix of a Hermitian matrix \(A\) is unitary when \(A\) has distinct eigenvalues.

Lemma 1: if the eigenvalues of a Hermitian matrix \(A\) are distinct, then there exists a unitary matrix \(U\) and a diagonal matrix \(D\) such that

\[ A U = U D. \]

Proof:

Will be added later.

With the column vectors of \(U\) the eigenvectors of \(A\) and the diagonal elements of \(D\) the corresponding eigenvalues of \(A\).

Theorem 9: let \(A\) be an \(n \times n\) matrix, there exists a unitary matrix \(U\) and a upper triangular matrix \(T\) such that

\[ A U = U T. \]

Proof:

Will be added later.

The factorization \(A = U T U^H\) is often referred to as the Schur decomposition of \(A\).

Theorem 10: if \(A\) is Hermitian, then there exists a unitary matrix \(U\) and a diagonal matrix \(D\) such that

\[ A U = U D. \]

Proof:

Will be added later.