Chapter 8: Differentiating Composite Functions (The Chain Rule)

Q: When is the chain rule used?

It is used for composite functions in which one function is nested inside another, such as (x^2+1)^10, e^(3x), and sin(2x). Backpropagation in machine learning also relies on repeated applications of the chain rule across the layers of a neural network.

Chain Rule

Beginner

8.1 What Is a Composite Function?

A function in which one function is applied to the result of another is called a composite function.

Definition of a Composite Function

The composite function of two functions $f$ and $g$ is defined by:

$$(f \circ g)(x) = f(g(x))$$

This means that we first evaluate $g$ and then use its result as the input to $f$.

Figure 1: The composite function $f(g(x))$. Applying $g$ to $x$ gives $g(x)$, and then $f$ is applied to it. The whole map is written $f \circ g$.

Example: Composite Functions

Let $f(x) = x^2$ and $g(x) = x + 1$. Then:

$$(f \circ g)(x) = f(g(x)) = f(x+1) = (x+1)^2$$ $$(g \circ f)(x) = g(f(x)) = g(x^2) = x^2 + 1$$

Note that, in general, $f \circ g \neq g \circ f$. Changing the order of composition usually changes the result.

Example: Decomposing a Composite Function

Consider $h(x) = (x^2 + 3x)^5$. We can decompose it as follows:

Inner function: $g(x) = x^2 + 3x$ (the expression inside)
Outer function: $f(u) = u^5$ (the operation applied to the inner expression)
$h(x) = f(g(x)) = (g(x))^5$

8.2 The Chain Rule

Theorem (Chain Rule)

If $g$ is differentiable at $x$ and $f$ is differentiable at $u=g(x)$, then the composite function $y=f(g(x))$ is differentiable at $x$, and:

$$\dfrac{dy}{dx} = \dfrac{dy}{du} \cdot \dfrac{du}{dx}$$

Equivalently, in function notation:

$$(f(g(x)))' = f'(g(x)) \cdot g'(x)$$

In words: the derivative of the outer function multiplied by the derivative of the inner function.

Proof

We begin with the definition of the derivative. Let $h(x) = f(g(x))$. Then:

$$h'(x) = \lim_{\Delta x \to 0} \dfrac{f(g(x + \Delta x)) - f(g(x))}{\Delta x}$$

Step 1: Let $\Delta u = g(x + \Delta x) - g(x)$.

As $\Delta x \to 0$, we have $\Delta u \to 0$ because $g$ is continuous.

Step 2: Let $u = g(x)$. When $\Delta u \ne 0$, multiply and divide by $\Delta u$:

$$\dfrac{f(g(x+\Delta x))-f(g(x))}{\Delta x}=\dfrac{f(u+\Delta u)-f(u)}{\Delta u}\cdot\dfrac{\Delta u}{\Delta x}$$

Step 3: As $\Delta x\to0$, we have $\Delta u\to0$, so the first fraction on the right approaches $f'(u)=f'(g(x))$ and the second approaches $g'(x)$. Therefore

$$h'(x)=f'(g(x))\,g'(x)$$

This proves the chain rule.

(Note) For some functions, there are $\Delta x$ with $\Delta u=0$ no matter how close $\Delta x$ is to $0$, and then the division in Step 2 is not possible. The chain rule still holds in that case, but a rigorous proof needs an extra idea (studied at university).

Figure 2: How the chain rule works. A change in $x$ passes through $u = g(x)$ to $y = f(u)$, and the rates of change multiply: $\dfrac{dy}{dx} = \dfrac{dy}{du} \cdot \dfrac{du}{dx}$.

8.3 Applying the Chain Rule

Example 1: Differentiating $(x^2 + 1)^3$

Inner function: $u = g(x) = x^2 + 1$; outer function: $y = f(u) = u^3$.

Step 1: Differentiate the outer function with respect to $u$.

$$\dfrac{dy}{du} = 3u^2$$

Step 2: Differentiate the inner function with respect to $x$.

$$\dfrac{du}{dx} = 2x$$

Step 3: Apply the chain rule.

$$\dfrac{dy}{dx} = \dfrac{dy}{du} \cdot \dfrac{du}{dx} = 3u^2 \cdot 2x$$

Step 4: Substitute the original expression for $u$.

$$= 3(x^2 + 1)^2 \cdot 2x = 6x(x^2 + 1)^2$$

Example 2: Differentiating $\sqrt{x^3 + 2x}$

Let $y = \sqrt{u} = u^{1/2}$ and $u = x^3 + 2x$.

Step 1: Differentiate the outer function.

$$\dfrac{dy}{du} = \dfrac{1}{2}u^{-1/2} = \dfrac{1}{2\sqrt{u}}$$

Step 2: Differentiate the inner function.

$$\dfrac{du}{dx} = 3x^2 + 2$$

Step 3: Apply the chain rule.

$$\dfrac{dy}{dx} = \dfrac{1}{2\sqrt{u}} \cdot (3x^2 + 2)$$

Step 4: Substitute the original expression for $u$.

$$= \dfrac{3x^2 + 2}{2\sqrt{x^3 + 2x}}$$

Example 3: Differentiating $\dfrac{1}{(x^2 + 1)^2}$

Let $y = u^{-2}$ and $u = x^2 + 1$.

Step 1: Differentiate the outer function.

$$\dfrac{dy}{du} = -2u^{-3} = -\dfrac{2}{u^3}$$

Step 2: Differentiate the inner function.

$$\dfrac{du}{dx} = 2x$$

Step 3: Apply the chain rule.

$$\dfrac{dy}{dx} = -\dfrac{2}{u^3} \cdot 2x = -\dfrac{4x}{(x^2 + 1)^3}$$

8.4 The General Form of the Chain Rule

General Power Rule for Composite Functions

The derivative of $[g(x)]^n$ is:

$$\dfrac{d}{dx}[g(x)]^n = n[g(x)]^{n-1} \cdot g'(x)$$

Derivation

Let $f(u) = u^n$ and $u = g(x)$. Applying the chain rule gives:

$$\dfrac{d}{dx}[g(x)]^n = f'(g(x)) \cdot g'(x) = n[g(x)]^{n-1} \cdot g'(x)$$

Figure 3: The general power rule $\dfrac{d}{dx}[g(x)]^n = n[g(x)]^{n-1} g'(x)$ and the example $(x^2+1)^3$.

8.5 Nested Composite Functions

The chain rule can also be applied to compositions of three or more functions.

The Generalized Chain Rule

For $y = f(g(h(x)))$:

$$\dfrac{dy}{dx} = f'(g(h(x))) \cdot g'(h(x)) \cdot h'(x)$$

Equivalently:

$$\dfrac{dy}{dx} = \dfrac{dy}{du} \cdot \dfrac{du}{dv} \cdot \dfrac{dv}{dx}$$

where $u = g(v)$ and $v = h(x)$.

Example: Differentiating $y = \sqrt{(x^2+1)^3}$

This is a composition of three functions:

Innermost function: $v = x^2 + 1$
Middle function: $u = v^3$
Outer function: $y = \sqrt{u} = u^{1/2}$

Step 1: Differentiate each layer.

$\dfrac{dy}{du} = \dfrac{1}{2}u^{-1/2}$
$\dfrac{du}{dv} = 3v^2$
$\dfrac{dv}{dx} = 2x$

Step 2: Apply the chain rule.

$$\dfrac{dy}{dx} = \dfrac{1}{2}u^{-1/2} \cdot 3v^2 \cdot 2x$$

Step 3: Substitute the original variables.

$$= \dfrac{1}{2}(v^3)^{-1/2} \cdot 3v^2 \cdot 2x$$ $$= \dfrac{1}{2} \cdot v^{-3/2} \cdot 3v^2 \cdot 2x$$ $$= 3x \cdot v^{-3/2+2} = 3x \cdot v^{1/2}$$ $$= 3x\sqrt{x^2+1}$$

8.6 Combining the Chain Rule with Other Rules

Example 1: Differentiating $y = x^2(x+1)^3$

Combine the product rule with the chain rule.

Let $f(x) = x^2$ and $g(x) = (x+1)^3$.

Differentiate Each Function

$f'(x) = 2x$
$g'(x) = 3(x+1)^2 \cdot 1 = 3(x+1)^2$ (chain rule)

Apply the Product Rule

$$y' = f'(x)g(x) + f(x)g'(x)$$ $$= 2x(x+1)^3 + x^2 \cdot 3(x+1)^2$$ $$= (x+1)^2[2x(x+1) + 3x^2]$$ $$= (x+1)^2(2x^2 + 2x + 3x^2)$$ $$= (x+1)^2(5x^2 + 2x)$$ $$= x(x+1)^2(5x + 2)$$

Example 2: Differentiating $y = \dfrac{(x-1)^2}{(x+1)^3}$

Combine the quotient rule with the chain rule.

Let $f(x) = (x-1)^2$ and $g(x) = (x+1)^3$.

Differentiate Each Function

$f'(x) = 2(x-1)$
$g'(x) = 3(x+1)^2$

Apply the Quotient Rule

$$y' = \dfrac{f'g - fg'}{g^2}$$ $$= \dfrac{2(x-1)(x+1)^3 - (x-1)^2 \cdot 3(x+1)^2}{(x+1)^6}$$

Factor Out the Common Terms

The common factor in the numerator is $(x-1)(x+1)^2$:

$$= \dfrac{(x-1)(x+1)^2[2(x+1) - 3(x-1)]}{(x+1)^6}$$ $$= \dfrac{(x-1)[2x+2-3x+3]}{(x+1)^4}$$ $$= \dfrac{(x-1)(5-x)}{(x+1)^4}$$ $$= -\dfrac{(x-1)(x-5)}{(x+1)^4}$$

8.7 Summary of the Chain Rule

Figure 4: Steps for applying the chain rule, with the example of differentiating $(x^2+3x)^5$.

Frequently Asked Questions

What is the intuitive meaning of the chain rule $h'(x) = f'(g(x)) \cdot g'(x)$?

It means the rate of change of the outer function multiplied by the rate of change of the inner function. When $x$ changes slightly, $u = g(x)$ changes at a rate of $g'(x)$, and then $y = f(u)$ changes at a rate of $f'(u)$. The overall rate of change is the product of these two rates.

When is the chain rule used?

It is used for all composite functions in which one function is nested inside another, such as $(x^2+1)^{10}$, $e^{3x}$, and $\sin(2x)$. Backpropagation in machine learning also relies on repeated applications of the chain rule across the layers of a neural network.

What is the most common mistake when applying the chain rule?

The most common mistake is differentiating only the outer function and forgetting the inner function. For example, $(\sin(x^2))' = \cos(x^2)$ is incomplete. We must also multiply by the derivative of the inner function $x^2$, which is $2x$. The correct answer is $\cos(x^2) \cdot 2x$. Always remember to include the derivative of the inner function.

Exercises

Exercise 1: Basic Applications of the Chain Rule

Differentiate the following functions.

$(2x+1)^4$
$(x^3-1)^5$
$\sqrt{3x+2}$
$\dfrac{1}{(x^2+1)^3}$

Exercise 2: Combined Applications

Differentiate the following functions.

$x(x+1)^4$
$(x-1)^2(x+2)^3$
$\dfrac{x}{(x+1)^2}$

Exercise 3: Nested Composite Functions

Differentiate $y = ((x+1)^2 + 1)^3$.

Show Answers

Answers to Exercise 1

$4(2x+1)^3 \cdot 2 = 8(2x+1)^3$
$5(x^3-1)^4 \cdot 3x^2 = 15x^2(x^3-1)^4$
$\dfrac{1}{2}(3x+2)^{-1/2} \cdot 3 = \dfrac{3}{2\sqrt{3x+2}}$
$-3(x^2+1)^{-4} \cdot 2x = -\dfrac{6x}{(x^2+1)^4}$

Answers to Exercise 2

Using the product rule:
$= 1 \cdot (x+1)^4 + x \cdot 4(x+1)^3$
$= (x+1)^3[(x+1) + 4x] = (x+1)^3(5x+1)$
Using the product rule:
$= 2(x-1)(x+2)^3 + (x-1)^2 \cdot 3(x+2)^2$
$= (x-1)(x+2)^2[2(x+2) + 3(x-1)]$
$= (x-1)(x+2)^2(5x+1)$
Using the quotient rule:
$= \dfrac{1 \cdot (x+1)^2 - x \cdot 2(x+1)}{(x+1)^4}$
$= \dfrac{(x+1) - 2x}{(x+1)^3} = \dfrac{1-x}{(x+1)^3}$

Answer to Exercise 3

Let $u = (x+1)^2 + 1$. Then $y = u^3$.

$\dfrac{dy}{du} = 3u^2$

$\dfrac{du}{dx} = 2(x+1)$

Therefore:

$\dfrac{dy}{dx} = 3u^2 \cdot 2(x+1) = 6(x+1)[(x+1)^2+1]^2$