Extrema

Definition: for \(D \subseteq \mathbb{R}^n\) let \(f: D \to \mathbb{R}\) be differentiable and \(D\) contains no boundary points (open). A point \(\mathbf{x^*} \in D\) is called a critical point for \(f\) \(\iff \nabla f(\mathbf{x^*}) = \mathbf{0}\).

Definition: \(f\) has (strict) global \(\begin{matrix} \text{ maximum } \\ \text{ minimum } \end{matrix}\) in \(\mathbf{x^*} \in D\) \(\iff \forall \mathbf{x} \in D \backslash \{\mathbf{x^*}\} \Big[f(\mathbf{x^*}) \begin{matrix} (>) \\ \geq \\ \leq \\ (<) \end{matrix} f(\mathbf{x}) \Big]\).

Definition: \(f\) has (strict) local \(\begin{matrix} \text{ maximum } \\ \text{ minimum } \end{matrix}\) in \(\mathbf{x^*} \in D\) \(\iff \exists r_{>0} \forall \mathbf{x} \in D \backslash \{\mathbf{x^*}\} \Big[f(\mathbf{x^*}) \begin{matrix} (>) \\ \geq \\ \leq \\ (<) \end{matrix} f(\mathbf{x}) \;\land\; (0) < \|\mathbf{x} - \mathbf{x^*}\| < r \Big]\)

Theorem: if \(f\) has local \(\begin{matrix} \text{ maximum } \\ \text{ minimum } \end{matrix}\) at \(\mathbf{x^*} \in D\) then \(\mathbf{x^*}\) is a critical point of for \(f\).

Proof:

Will be added later.

A second derivative test

Definition: suppose \(f: \mathbb{R}^n \to \mathbb{R}\) is differentiable with \(\mathbf{x} \in \mathbb{R}^n\). The Hessian matrix of \(f\) is defined as

\[ H_f(\mathbf{x}) := \begin{pmatrix} \partial_{11} f(\mathbf{x}) & \dots & \partial_{1n} f(\mathbf{x}) \\ \vdots & \ddots & \vdots \\ \partial_{n1} f(\mathbf{x}) & \dots & \partial_{nn} f(\mathbf{x}) \end{pmatrix}. \]

Theorem:

• If \(H_f(\mathbf{x^*})\) is positive definite (all eigenvalues are positive), then \(f\) has a local minimum at \(\mathbf{x^*}\).
• If \(H_f(\mathbf{x^*})\) is negative definite (all eigenvalues are negative), then \(f\) has a local maximum at \(\mathbf{x^*}\).
• If \(H_f(\mathbf{x^*})\) is indefinite (both positive and negative eigenvalues), then \(f\) has a saddle point at \(\mathbf{x^*}\).
• If \(H_f(\mathbf{x^*})\) is neither positive nor negative definite, nor indefinite, (eigenvalues equal to zero) this test gives no information.

Proof:

Will be added later.

Extrema on restricted domains

Theorem: let \(D \subseteq \mathbb{R}^n\) be bounded and closed (\(D\) contains all boundary points). Let \(f: D \to \mathbb{R}\) be continuous, then \(f\) has a global maximum and minimum.

Proof:

Will be added later.

Procedure to find the global maximum and minimum:

• Find critical points in the interior.
• Find global extrema on the boundary.
• Find the largest/smallest among them.

Lagrange multipliers

Theorem: let \(f: M \to \mathbb{R}\) and \(g: \mathbb{R}^n \to \mathbb{R}\) with \(M\) the boundary of \(D\) given by

\[ M := \big\{\mathbf{x} \in \mathbb{R}^n \;\big|\; g(\mathbf{x}) = 0 \big\} \subseteq D, \]

suppose that there is global maximum or minimum \(\mathbf{x^*} \in M\) of \(f\) that is not an endpoint of \(M\) and \(\nabla g(\mathbf{x^*}) \neq \mathbf{0}\). Then there exists a \(\lambda^* \in \mathbb{R}\) such that \((\mathbf{x^*}, \lambda^*)\) is a critical point of the Lagrange function

\[ L(\mathbf{x}, \lambda) := f(\mathbf{x}) - \lambda g(\mathbf{x}). \]

Proof:

Will be added later.

The general case

Theorem: Let \(f: S \to \mathbb{R}\) and \(\mathbf{g}: \mathbb{R}^m \to \mathbb{R}^n\) with \(m \leq n -1\) restrictions given by

\[ S := \big\{\mathbf{x} \in \mathbb{R}^n \;\big|\; \mathbf{g}(\mathbf{x}) = 0 \big\} \subseteq D, \]

suppose that there is global maximum or minimum \(\mathbf{x^*} \in S\) of \(f\) that is not an endpoint of \(S\) and \(D \mathbf{g}(\mathbf{x^*}) \neq \mathbf{0}\). Then there exists a \(\mathbf{\lambda^*} \in \mathbb{R^m}\) such that \((\mathbf{x^*}, \mathbf{\lambda^*})\) is a critical point of the Lagrange function

\[ L(\mathbf{x}, \mathbf{\lambda}) := f(\mathbf{x}) - \big\langle \mathbf{\lambda},\; \mathbf{g}(\mathbf{x}) \big\rangle. \]

Proof:

Will be added later.

Example

Let \(f: M_1 \cap M_2 \to \mathbb{R}\) and \(g_{1,2}: \mathbb{R}^n \to \mathbb{R}\) with the restrictions given by

\[ M_{1,2} := \big\{\mathbf{x} \in \mathbb{R}^n \;\big|\; g_{1,2}(\mathbf{x}) = 0 \big\} \subseteq D, \]

suppose that there is global maximum or minimum \(\mathbf{x^*} \in M_1 \cap M_2\) of \(f\) that is not an endpoint of \(M_1 \cap M_2\) and \(\nabla g_{1,2}(\mathbf{x^*}) \neq \mathbf{0}\). Then there exists a \(\lambda_{1,2}^* \in \mathbb{R}\) such that \((\mathbf{x^*}, \lambda_{1,2}^*)\) is a critical point of the Lagrange function

\[ L(\mathbf{x}, \lambda_1, \lambda_2) := f(\mathbf{x}) - \lambda_1 g_1(\mathbf{x}) - \lambda_2 g_2(\mathbf{x}). \]