What is the difference between a gradient and a gradient flow?

The gradient ∇f is a vector that points in the direction of steepest increase of the function value (the uphill direction). The gradient flow is the opposite: a continuous-time system that flows along −∇f, the direction of steepest decrease. It is the continuous-time analogue of gradient descent in optimization.

How does this demo compute the gradient?

Partial derivatives are approximated by central differences: ∂f/∂x ≈ ( f(x+h, y) − f(x−h, y) ) / (2h), and similarly for ∂f/∂y. Central differences have smaller truncation error than forward differences and let us draw the gradient field of an arbitrary function without supplying any analytical derivative formula.

Why do trajectories appear to stall on the Rosenbrock function?

The Rosenbrock function has a long, curved, narrow valley. The gradient is steep across the valley walls but becomes extremely small along the floor of the valley. Because of this anisotropy, particles fall quickly into the valley but then advance only slowly along its floor, so their trajectories cluster densely and appear to stall. This is the same phenomenon that makes plain gradient descent slow on ill-conditioned problems.

Gradient Flow Visualization

Interactive demo of two-variable gradient flow via central-difference numerical differentiation

An interactive demo that visualizes the gradient field of a two-variable function $f(x,y)$. Click anywhere on the screen to place a particle: it flows along the direction opposite to the gradient (the steepest-descent direction). This provides an intuitive view of the trajectory of gradient descent.

Negative gradient vectors

Hover the mouse to display
coordinates and gradient

What Is a Gradient Flow?

Gradient vs. Gradient Flow

The gradient $\nabla f$ is a vector pointing in the direction in which the function value increases most steeply (the uphill direction). The gradient flow, in contrast, is its opposite: it flows along $-\nabla f$ (the downhill direction). Note the minus sign. Physically, it corresponds to an object rolling down in the direction in which potential energy decreases, and in optimization it is the continuous-time counterpart of gradient descent.

The ODE of Gradient Flow

The gradient flow obeys the following ordinary differential equation:

$$\dfrac{d\mathbf{x}}{dt} = -\nabla f(\mathbf{x})$$

The particle always moves in the direction in which the function value decreases most steeply, and stops upon reaching a local minimum where $\nabla f = \mathbf{0}$.

Gradient Computation via Numerical Differentiation

This demo approximates the gradient using central differences:

$$\begin{aligned} \dfrac{\partial f}{\partial x} &\approx \dfrac{f(x+h, y) - f(x-h, y)}{2h} \\[6pt] \dfrac{\partial f}{\partial y} &\approx \dfrac{f(x, y+h) - f(x, y-h)}{2h} \end{aligned}$$

Points to Observe

Basin of Attraction — For functions with multiple local minima, different initial positions converge to different minima. Note which valley each trajectory from a grid point is drawn into, and observe the structure of the boundaries between basins.
Saddle Point — In saddle-point functions, an unstable equilibrium appears at which the gradient is zero but the point is not a minimum. Particles pause briefly near the saddle and then diverge rapidly.
Narrow Valley — In the Rosenbrock function, particles descend steeply into the floor of the valley from outside, but along the floor the gradient becomes extremely small, so trajectories cluster densely and stall.