Inner product spaces

Definition

An introduction of length in a vector space may be formulated in terms of an inner product space.

Definition 1: an inner product on \(V\) is an operation on \(V\) that assigns, to each pair of vectors \(\mathbf{x},\mathbf{y} \in V\), a real number \(\langle \mathbf{x},\mathbf{y}\rangle\) satisfying the following conditions

1. \(\langle \mathbf{x},\mathbf{x}\rangle > 0, \text{ for } \mathbf{x} \in V\backslash\{\mathbf{0}\} \text{ and } \langle \mathbf{x},\mathbf{x}\rangle = 0, \; \text{for } \mathbf{x} = \mathbf{0}\),
2. \(\langle \mathbf{x},\mathbf{y}\rangle = \overline{\langle \mathbf{y},\mathbf{x}\rangle}, \; \forall \mathbf{x}, \mathbf{y} \in V\),
3. \(\langle a \mathbf{x} + b \mathbf{y}, \mathbf{z}\rangle = a \langle \mathbf{x},\mathbf{z}\rangle + b \langle \mathbf{y},\mathbf{z}\rangle, \; \forall \mathbf{x}, \mathbf{y}, \mathbf{z} \in V \text{ and } a,b \in \mathbb{K}\).

A vector space \(V\) with an inner product is called an inner product space.

Euclidean inner product spaces

The standard inner product on the Euclidean vector spaces \(V = \mathbb{R}^n\) with \(n \in \mathbb{N}\) is given by the scalar product defined by

\[ \langle \mathbf{x},\mathbf{y}\rangle = \mathbf{x}^T \mathbf{y}, \]

for all \(\mathbf{x},\mathbf{y} \in V\).

Proof:

Will be added later.

This can be extended to matrices \(V = \mathbb{R}^{m \times n}\) with \(m,n \in \mathbb{N}\) for which an inner product may be given by

\[ \langle A, B\rangle = \sum_{i=1}^m \sum_{j=1}^n a_{ij} b_{ij}, \]

for all \(A, B \in V\).

Proof:

Will be added later.

Function inner product spaces

Let \(V\) be a function space with a domain \(X\). An inner product on \(V\) may be defined by

\[ \langle f, g\rangle = \int_X \bar f(x) g(x) dx \]

for all \(f,g \in V\).

Proof:

Will be added later.

Polynomial inner product spaces

Let \(V\) be a polynomial space of degree \(n \in \mathbb{N}\) with the set of numbers \(\{x_i\}_{i=1}^n \subset \mathbb{K}^n\). An inner product on \(V\) may be defined by

\[ \langle p, q \rangle = \sum_{i=1}^n \bar p(x_i) q(x_i), \]

for all \(p,q \in V\).

Proof:

Will be added later.

Properties of inner product spaces

Definition 2: let \(V\) be an inner product space, the Euclidean length \(\|\mathbf{v}\|\) of a vector \(\mathbf{v}\) is defined as

\[ \|\mathbf{v}\| = \sqrt{\langle \mathbf{v}, \mathbf{v} \rangle}, \]

for all \(\mathbf{v} \in V\).

Which is consistent with Euclidean geometry. According to definition 1 the distance between two vectors \(\mathbf{v}, \mathbf{w} \in V\) is \(\|\mathbf{v} - \mathbf{w}\|\).

Definition 3: let \(V\) be an inner product space, the vectors \(\mathbf{u}\) and \(\mathbf{v}\) are orthogonal if

\[ \langle \mathbf{u}, \mathbf{v} \rangle = 0, \]

for all \(\mathbf{u}, \mathbf{v} \in V\).

A pair of orthogonal vectors will satisfy the theorem of Pythagoras.

Theorem 1: let \(V\) be an inner product space and \(\mathbf{u}\) and \(\mathbf{v}\) are orthogonal then

\[ \|\mathbf{u} + \mathbf{v}\|^2 = \|\mathbf{u}\|^2 + \|\mathbf{v}\|^2, \]

for all \(\mathbf{u}, \mathbf{v} \in V\).

Proof:

let \(V\) be an inner product space and let \(\mathbf{u}, \mathbf{v} \in V\) be orthogonal, then

\[ \begin{align*} \|\mathbf{u} + \mathbf{v}\|^2 &= \langle \mathbf{u} + \mathbf{v}, \mathbf{u} + \mathbf{v}\rangle, \\ &= \langle \mathbf{u}, \mathbf{u} \rangle + 2 \langle \mathbf{u}, \mathbf{v} \rangle + \langle \mathbf{v}, \mathbf{v} \rangle, \\ &= \|\mathbf{u}\|^2 + \|\mathbf{v}\|^2. \end{align*} \]

Interpreted in \(\mathbb{R}^2\) this is just the familiar Pythagorean theorem.

Definition 4: let \(V\) be an inner product space then the scalar projection \(a\) of \(\mathbf{u}\) onto \(\mathbf{v}\) is defined as

\[ a = \frac{1}{\|\mathbf{v}\|} \langle \mathbf{u}, \mathbf{v} \rangle, \]

for all \(\mathbf{u} \in V\) and \(\mathbf{v} \in V \backslash \{\mathbf{0}\}\).

The vector projection \(p\) of \(\mathbf{u}\) onto \(\mathbf{v}\) is defined as

\[ \mathbf{p} = a \bigg(\frac{1}{\|\mathbf{v}\|} \mathbf{v}\bigg) = \frac{\langle \mathbf{u}, \mathbf{v} \rangle}{\langle \mathbf{v}, \mathbf{v} \rangle} \mathbf{v}, \]

for all \(\mathbf{u} \in V\) and \(\mathbf{v} \in V \backslash \{\mathbf{0}\}\).

It may be observed that \(\mathbf{u} - \mathbf{p}\) and \(\mathbf{p}\) are orthogonal since \(\langle \mathbf{p}, \mathbf{p} \rangle = a^2\) and \(\langle \mathbf{u}, \mathbf{p} \rangle = a^2\) which implies

\[ \langle \mathbf{u} - \mathbf{p}, \mathbf{p} \rangle = \langle \mathbf{u}, \mathbf{p} \rangle - \langle \mathbf{p}, \mathbf{p} \rangle = a^2 - a^2 = 0. \]

Additionally, it may be observed that \(\mathbf{u} = \mathbf{p}\) if and only if \(\mathbf{u}\) is a scalar multiple of \(\mathbf{v}\); \(\mathbf{u} = b \mathbf{v}\) for some \(b \in \mathbb{K}\). Since

\[ \mathbf{p} = \frac{\langle b \mathbf{v}, \mathbf{v} \rangle}{\langle \mathbf{v}, \mathbf{v} \rangle} \mathbf{v} = b \mathbf{v} = \mathbf{u}. \]

Theorem 2: let \(V\) be an inner product space then

\[ | \langle \mathbf{u}, \mathbf{v} \rangle | \leq \| \mathbf{u} \| \| \mathbf{v} \|, \]

is true for all \(\mathbf{u}, \mathbf{v} \in V\). With equality only holding if and only if \(\mathbf{u}\) and \(\mathbf{v}\) are linearly dependent.

Proof:

let \(V\) be an inner product space and let \(\mathbf{u}, \mathbf{v} \in V\). If \(\mathbf{v} = \mathbf{0}\), then

\[ | \langle \mathbf{u}, \mathbf{v} \rangle | = 0 = \| \mathbf{u} \| \| \mathbf{v} \|, \]

If \(\mathbf{v} \neq \mathbf{0}\), then let \(\mathbf{p}\) be the vector projection of \(\mathbf{u}\) onto \(\mathbf{v}\). Since \(\mathbf{p}\) is orthogonal to \(\mathbf{u} - \mathbf{p}\) it follows that

\[ \| \mathbf{p} \|^2 + \| \mathbf{u} - \mathbf{p} \|^2 = \| \mathbf{u} \|^2, \]

thus

\[ \frac{1}{\|\mathbf{v}\|^2} \langle \mathbf{u}, \mathbf{v} \rangle^2 = \| \mathbf{p}\|^2 = \| \mathbf{u} \|^2 - \| \mathbf{u} - \mathbf{p} \|^2, \]

and hence

\[ \langle \mathbf{u}, \mathbf{v} \rangle^2 = \|\mathbf{u}\|^2 \|\mathbf{v}\|^2 - \|\mathbf{u} - \mathbf{p}\|^2 \|\mathbf{v}\|^2 \leq \|\mathbf{u}\|^2 \|\mathbf{v}\|^2, \]

therefore

\[ | \langle \mathbf{u}, \mathbf{v} \rangle | \leq \| \mathbf{u} \| \| \mathbf{v} \|. \]

Equality holds if and only if \(\mathbf{u} = \mathbf{p}\). From the above observations, this condition may be restated to linear dependence of \(\mathbf{u}\) and \(\mathbf{v}\).

A consequence of the Cauchy-Schwarz inequality is that if \(\mathbf{u}\) and \(\mathbf{v}\) aer nonzero vectors in an inner product space then

\[ -1 \leq \frac{\langle \mathbf{u}, \mathbf{v} \rangle}{\|\mathbf{u}\| \|\mathbf{v}\|} \leq 1, \]

and hence there is a unique angle \(\theta \in [0, \pi]\) such that

\[ \cos \theta = \frac{\langle \mathbf{u}, \mathbf{v} \rangle}{\|\mathbf{u}\| \|\mathbf{v}\|}. \]

Normed spaces

Definition 5: a vector space \(V\) is said to be a normed linear space if to each vector \(\mathbf{v} \in V\) there is associated a real number \(\| \mathbf{v} \|\) satisfying the following conditions

1. \(\|\mathbf{v}\| > 0, \text{ for } \mathbf{v} \in V\backslash\{\mathbf{0}\} \text{ and } \| \mathbf{v} \| = 0, \text{ for } \mathbf{v} = \mathbf{0}\),
2. \(\|a \mathbf{v}\| = |a| \|\mathbf{v}\|, \; \forall \mathbf{v} \in V \text{ and } a \in \mathbb{K}\),
3. \(\| \mathbf{v} + \mathbf{w}\| \geq \|\mathbf{v}\| + \| \mathbf{w}\|, \; \forall \mathbf{v}, \mathbf{w} \in V\),

is called the norm of \(\mathbf{v}\).

With the third condition, the triangle inequality.

Theorem 3: let \(V\) be an inner product space then

\[ \| \mathbf{v} \| = \sqrt{\langle \mathbf{v}, \mathbf{v} \rangle}, \]

for all \(\mathbf{v} \in V\) defines a norm on \(V\).

Proof:

Will be added later.

We therefore have that the Euclidean length (definition 2) is a norm, justifying the notation.