CAL1_L17

The chain rule

Theorem 1(Theorem).

Assume that $\mathbf{g}$ is differentiable at $\mathbf{a}$, with total derivative ${\mathbf{g}}^{\prime }(\mathbf{a})$. Let $\mathbf{b}=\mathbf{g}(\mathbf{a})$ and assume that $\mathbf{f}$ is differentiable at $\mathbf{b}$, with total derivative ${\mathbf{f}}^{\prime }(\mathbf{b})$. Then the composition function $\mathbf{h}=\mathbf{f}\circ \mathbf{g}$ is differentiable at $\mathbf{a}$ and the total derivative ${\mathbf{h}}^{\prime }(\mathbf{a})$ is given by

$${\mathbf{h}}^{\prime }(\mathbf{a})={\mathbf{f}}^{\prime }(\mathbf{b})\circ {\mathbf{g}}^{\prime }(\mathbf{a}),$$

the composition of the linear functions ${\mathbf{f}}^{\prime }(\mathbf{b})$ and ${\mathbf{g}}^{\prime }(\mathbf{a})$.

Proof
We will show that a first order Taylor formula exists for $\mathbf{h}$.

$$\begin{array}{rl}\mathbf{h}(\mathbf{a}+\mathbf{y})-\mathbf{h}(\mathbf{a})& =\mathbf{f}(\mathbf{g}(\mathbf{a}+\mathbf{y}))-\mathbf{f}(\mathbf{g}(\mathbf{a}))\\ & =\mathbf{f}(\mathbf{b}+\mathbf{v})-\mathbf{f}(\mathbf{b}),\end{array}$$

where $\mathbf{v}\equiv \mathbf{g}(\mathbf{a}+\mathbf{y})-\mathbf{b}$. Since $\mathbf{g}$ is differentiable at $\mathbf{a}$, we have

$$\begin{array}{rlr}\mathbf{v}={\mathbf{g}}^{\prime }(\mathbf{a})(\mathbf{y})+\Vert \mathbf{y}\Vert {\mathbf{E}}_{\mathbf{a}}(\mathbf{y}),& & (1)\end{array}$$

where ${\mathbf{E}}_{\mathbf{a}}(\mathbf{y})\to 0$ as $\mathbf{y}\to 0$. Since $\mathbf{f}$ is differentiable at $\mathbf{b}$, we have

$$\begin{array}{rl}\mathbf{f}(\mathbf{b}+\mathbf{v})-\mathbf{f}(\mathbf{b})={\mathbf{f}}^{\prime }(\mathbf{b})& (\mathbf{v})+\Vert \mathbf{v}\Vert {\mathbf{E}}_{\mathbf{b}}(\mathbf{v}),\end{array}$$

where ${\mathbf{E}}_{\mathbf{b}}(\mathbf{v})\to 0$ as $\mathbf{v}\to 0$. Using $(1)$,

$$\begin{array}{rl}\mathbf{f}(\mathbf{b}+\mathbf{v})-\mathbf{f}(\mathbf{b})& ={\mathbf{f}}^{\prime }(\mathbf{b})[{\mathbf{g}}^{\prime }(\mathbf{a})(\mathbf{y})]+\Vert \mathbf{y}\Vert {\mathbf{f}}^{\prime }(\mathbf{b})[{\mathbf{E}}_{\mathbf{a}}(\mathbf{y})]+\Vert \mathbf{v}\Vert {\mathbf{E}}_{\mathbf{b}}(\mathbf{v})\\ & ={\mathbf{f}}^{\prime }(\mathbf{b})[{\mathbf{g}}^{\prime }(\mathbf{a})(\mathbf{y})]+\Vert \mathbf{y}\Vert \mathbf{E}(\mathbf{y}),\end{array}$$

where $\mathbf{E}(0)=0$ and

$$\mathbf{E}(\mathbf{y})={\mathbf{f}}^{\prime }(\mathbf{b})[{\mathbf{E}}_{\mathbf{a}}(\mathbf{y})]+\frac{\Vert \mathbf{v}\Vert }{\Vert \mathbf{y}\Vert }{\mathbf{E}}_{\mathbf{b}}(\mathbf{v})$$

if $\mathbf{y}\ne 0$. To complete the proof we need to show that $\mathbf{E}(\mathbf{y})\to 0$ as $\mathbf{y}\to 0$. The first term clearly tends to $0$ as $\mathbf{y}\to 0$, and ${\mathbf{E}}_{\mathbf{b}}(0)\to 0$ since $\mathbf{v}\to 0$ as $\mathbf{y}\to 0$. We will be done if we show that $\Vert \mathbf{v}\Vert /\Vert \mathbf{y}\Vert$ is bounded. From results obtained here,

$$\begin{array}{r}\Vert \mathbf{v}\Vert \le \Vert {\mathbf{g}}^{\prime }(\mathbf{a})(\mathbf{y})\Vert +\Vert \mathbf{y}\Vert \Vert {\mathbf{E}}_{\mathbf{a}}(\mathbf{y})\Vert \le \Vert \mathbf{y}\Vert (M+\Vert {\mathbf{E}}_{\mathbf{a}}(\mathbf{y})\Vert ),\end{array}$$

where $M={\sum }_{k=1}^m\Vert \nabla g_k(\mathbf{a})\Vert$. Hence $\Vert \mathbf{v}\Vert /\Vert \mathbf{y}\Vert$ remains bounded as $\mathbf{y}\to 0$. We therefore obtain the first order Taylor formula

$$\begin{array}{rl}\mathbf{h}(\mathbf{a}+\mathbf{y})-\mathbf{h}(\mathbf{a})& ={\mathbf{f}}^{\prime }(\mathbf{b})[{\mathbf{g}}^{\prime }(\mathbf{a})(\mathbf{y})]+\Vert \mathbf{y}\Vert \mathbf{E}(\mathbf{y})\end{array}$$

where $\mathbf{E}(\mathbf{y})\to 0$ as $\mathbf{y}\to 0$. Thus, $\mathbf{h}$ is differentiable at $\mathbf{a}$, with derivative ${\mathbf{f}}^{\prime }(\mathbf{b})\circ {\mathbf{g}}^{\prime }(\mathbf{a})$.

The matrix of ${\mathbf{h}}^{\prime }(\mathbf{a})$ is given by

$$\mathbf{Dh}(\mathbf{a})=\mathbf{Df}(\mathbf{b})\mathbf{Dg}(\mathbf{a}).$$

If $\mathbf{g}:{\mathbb{R}}^p\to {\mathbb{R}}^n$ and $\mathbf{f}:{\mathbb{R}}^n\to {\mathbb{R}}^m$, then $\mathbf{Dg}(\mathbf{a})$ is an $n\times p$ matrix, $\mathbf{Df}(\mathbf{b})$ is an $m\times n$ matrix, and $\mathbf{D}(\mathbf{ha})$ is an $m\times p$ matrix. The above matrix equation is equivalent to the $mp$ scalar equations

$$\begin{array}{rl}{D}_j{h}_i(\mathbf{a})& =\sum _{k=1}^n{D}_k{f}_i(\mathbf{b})D_j{g}_k(\mathbf{a})\\ & =\nabla f_i(\mathbf{b})\cdot D_j\mathbf{g}(\mathbf{a}).\end{array}$$

In particular, if $p=m=1$, $h^{\prime }(a)=\nabla f(\mathbf{b})\cdot \mathbf{Dg}(a)$.

Theorem 2(Theorem).

Let $f$ and $D_{2}f$ be continuous on a rectangle $[a,b]\times [c,d]$. Let $p$ and $q$ be differentiable on $[c,d]$ such that $p(y),q(y)\in [a,b]$ for each $y\in [c,d]$. Define $F$ by

$$F(y)\equiv {\int }_{p(y)}^{q(y)}f(x,y)dx,y\in [c,d].$$

Then $F^{\prime }(y)$ exists for each $y\in (c,d)$ and is given by

$$F^{\prime }(y)={\int }_{p(y)}^{q(y)}{D}_{2}f(x,y)dx+f(q(y),y)q^{\prime }(y)-f(p(y),y)p^{\prime }(y).$$

Note that continuous functions are integrable, and hence the above integrals are well defined.

Proof
Consider the maps $\mathbf{h}:[c,d]\to [a,b]\times [a,b]\times [c,d]$ and $g:[a,b]\times [a,b]\times [c,d]\to \mathbb{R}$ defined by $y\mapsto {\left[\begin{array}{ccc}p(y)& q(y)& y\end{array}\right]}^{⊺}$ and $\left[\begin{array}{ccc}{x}_1& x_2& x_3\end{array}\right]\mapsto {\int }_{x_1}^{x_2}f(t,x_3)dt$. Then, $F=g\circ \mathbf{h}$, and

$$\begin{array}{rl}{F}^{\prime }(y)& =\nabla g(\mathbf{h}(y))\cdot \mathbf{Dh}(y)\\ & =D_{1}g(\mathbf{h}(y))p^{\prime }(y)+D_{2}g(\mathbf{h}(y))q^{\prime }(y)+D_{3}g(\mathbf{h}(y))\end{array}$$

We know from here that

$$\begin{array}{rl}{D}_{1}g(\mathbf{h}(y))& =\left(\frac{d}{d{x}_1}{\int }_{x_1}^{x_2}f(t,x_3)dt\right)(\mathbf{h}(y))\\ & =(-f(x_1,x_3))(\mathbf{h}(y))\\ & =-f(p(y),y),\\ & \\ D_{2}g(\mathbf{h}(y))& =(f(x_2,x_3))(\mathbf{h}(y))\\ & =f(q(y),y).\end{array}$$

Also,

$$\begin{array}{rl}{D}_{3}g(\mathbf{h}(y))& =\left({\int }_{x_1}^{x_2}{D}_{2}f(t,x_3)dt\right)(\mathbf{h}(y))\\ & ={\int }_{p(y)}^{q(y)}{D}_{2}f(t,y)dt.\end{array}$$