CAL1_L24

Extremum problems with side conditions

Theorem 1(Theorem).

Let $f$ be a real valued function such that $f\in C^{\prime }$ on an open set $S$ in ${\mathbb{R}}^n$. Let $g_1,\dots ,g_m$ be $m$ real valued functions such that $\mathbf{g}=(g_1,\dots ,g_m)\in C_1$ on $S$, and assume that $m\le n$. Let $X_0$ be that subset of $S$ on which $\mathbf{g}$ vanishes, that is,

$$X_0=\{\mathbf{x}|\mathbf{x}\in S,\mathbf{g}(\mathbf{x})=0\}.$$

Assume that ${\mathbf{x}}_0\in X_0$ and assume that there exists an $n$-ball $B({\mathbf{x}}_0)$ such that $f(\mathbf{x})\le f({\mathbf{x}}_0)$ for all $\mathbf{x}\in X_0\cap B({\mathbf{x}}_0)$ or such that $f(\mathbf{x})\ge f({\mathbf{x}}_0)$ for all $\mathbf{x}\in X_0\cap B({\mathbf{x}}_0)$. Assume also that the $m$-rowed determinant $[D_j{g}_i({\mathbf{x}}_0)]\ne 0$. Then there exist $m$ real numbers ${\lambda }_1,\dots ,{\lambda }_m$ such that the following $n$ equations are satisfied:

$$D_{r}f({\mathbf{x}}_0)+\sum _{k=1}^m{\lambda }_k{D}_r{g}_k({\mathbf{x}}_0)=0(r=1,\dots ,n).$$

[!Example]
Let $f:{\mathbb{R}}^n\to \mathbb{R}$ be a symmetric quadratic form given by

$$f(x_1,\dots ,x_n)=\sum _{1\le i,j\le n}{a}_{i,j}{x}_i{x}_j.$$

Show that the maximum value of $f$ on $\{x\in {\mathbb{R}}^n|\Vert \mathbf{x}\Vert =1\}$ are eigenvalues of the matrix $[a_{i,j}{]}_{1\le i,j\le n}$.

Here, only a single constraint is provided: $g(\mathbf{x})=\Vert \mathbf{x}\Vert -1=0$. Note that the level set $L=\{x\in {\mathbb{R}}^n|g(\mathbf{x})=0\}$ is compact in ${\mathbb{R}}^n$. $f$ is a continuous function, so the restriction of $f$ to $L$ is continuous. From the extreme value theorem, $f$ must attain a maximum and a minimum value on $L$. Now,

$$\begin{array}{rl}{f}^{\prime }(\mathbf{x})& ={\left[\begin{array}{c}{\sum }_{i=1}^n{a}_{1,i}{x}_i+{\sum }_{i=1}^n{a}_{i,1}{x}_i\\ ⋮\\ {\sum }_{i=1}^n{a}_{n,i}{x}_i+{\sum }_{i=1}^n{a}_{i,n}{x}_i\end{array}\right]}^T\\ & =(A\mathbf{x}+A^T\mathbf{x}{)}^T\\ & =(2A\mathbf{x}{)}^T.\\ & \\ & \\ g^{\prime }(x)& =2{\mathbf{x}}^T\end{array}$$

Clearly, $f$ is a $C^1$ mapping, and so is $g$. For $x_1\ne 0$, $\det [2x_1]\ne 0$. Let $\mathbf{p}$ be the point on $L$ where $f$ attains its maximum value. Then, there exists $\lambda$ such that

$$\begin{array}{rl}& 2A\mathbf{p}=\lambda 2\mathbf{p}\\ ⟹& A\mathbf{p}=\lambda \mathbf{p}\end{array}$$

So, $\lambda$ is an eigenvalue of $A$. Now,

$$f(\mathbf{p})={\mathbf{p}}^{T}A\mathbf{p}=\lambda {\mathbf{p}}^T\mathbf{p}=\lambda .$$