Continuously Differentiable Functions are Locally Lipschitz
It turns out that our existence proof will actually hinge on our function satisfying a Lipschitz condition. So let’s show that we will have this property anyway.
More specifically, we are given a function defined on an open region . We want to show that around any point we have some neighborhood where satisfies a Lipschitz condition. That is: for and in the neighborhood , there is a constant and we have the inequality
We don’t have to use the same for each neighborhood, but every point should have a neighborhood with some .
where is the operator norm of . What this lemma says is that if the function is we can make this work out over finite distances, not just for infinitesimal displacements.
So, given our point let be the closed ball of radius around , and choose so small that is contained within . Since the function — which takes points to the space of linear operators — is continuous by our assumption, the function is continuous as well. The extreme value theorem tells us that since is compact this continuous function must attain a maximum, which we call .
The ball is also “convex”, meaning that given points and in the ball the whole segment for is contained within the ball. We define a function and use the chain rule to calculate
Then we calculate
And from this we conclude
That is, the separation between the outputs is expressible as an integral, the integrand of which is bounded by our infinitesimal result above. Integrating up we get the bound we seek.