2008 October « The Unapologetic Mathematician

Sine and Cosine

Now I want to consider the differential equation $f''(x)+f(x)=0$ . As I mentioned at the end of last time, we can write this as $f''(x)=(-1)f(x)$ and find two solutions — $\exp(ix)$ and $\exp(-ix)$ — by taking the two complex square roots of $-1$ . But the equation doesn’t use any complex numbers. Surely we can find real-valued functions that work.

Indeed, we can, and we’ll use the same techniques as we did before. We again find that any solution must be infinitely differentiable, and so we will assume that it’s analytic. Thus we write

$\displaystyle f(x)=\sum\limits_{k=0}^\infty a_kx^k$

and we take the first two derivatives

$\displaystyle f'(x)=\sum\limits_{k=0}^\infty(k+1)a_{k+1}x^k$
$\displaystyle f''(x)=\sum\limits_{k=0}^\infty(k+2)(k+1)a_{k+2}x^k$

The equation then reads

$\displaystyle a_{k+2}=-\frac{a_k}{(k+2)(k+1)}$

for every natural number $k$ . The values $a_0=f(0)$ and $a_1=f'(0)$ are not specified, and we can use them to set initial conditions.

We pick two sets of initial conditions to focus on. In the first case, $f(0)=0$ and $f'(0)=1$ , while in the second case $f(0)=1$ and $f'(0)=0$ . We call these two solutions the “sine” and “cosine” functions, respectively, writing them as $\sin(x)$ and $\cos(x)$ .

Let’s work out the series for the cosine function. We start with $a_1=0$ , and the recurrence relation tells us that all the odd terms will be zero. So let’s just write out the even terms $a_{2k}$ . First off, $a_0=1$ . Then to move from $a_{2k}$ to $a_{2k+2)}$ we multiply by $\frac{-1}{(2k+1)(2k+2)}$ . So in moving from $a_0$ all the way to $a_{2k}$ we’ve multiplied by $-1$ $k$ times, and we’ve multiplied up every number from ${1}$ to $2k$ . That is, we have $a_{2k}=\frac{(-1)^k}{(2k)!}$ , and we have the series

$\displaystyle\cos(x)=\sum\limits_{k=0}^\infty\frac{(-1)^kx^{2k}}{(2k)!}$

This isn’t the usual form for a power series, but it’s more compact than including all the odd terms. A similar line of reasoning leads to the following series expansion for the sine function:

$\displaystyle\sin(x)=\sum\limits_{k=0}^\infty\frac{(-1)^kx^{2k+1}}{(2k+1)!}$

Any other solution with $f(0)=a$ and $f'(0)=b$ then can be written as $a\cos(x)+b\sin(x)$ .

In particular, consider the first solutions we found above: $f(x)=\exp(ix)$ and $f(x)=\exp(-ix)$ . Each of them has $f(0)=1$ , and $f'(0)=\pm i$ , depending on which solution we pick. That is, we can write $\exp(ix)=\cos(x)+i\sin(x)$ , and $\exp(-ix)=\cos(x)-i\sin(x)$ .

Of course, the second of these equations is just the complex conjugate of the first, and so it’s unsurprising. The first, however, is called “Euler’s formula”, because it was proved by Roger Cotes. It’s been seen as particularly miraculous, but this is mostly because people’s first exposure to the sine and cosine functions usually comes from a completely different route, and the relationship between exponentials and trigonometry seems utterly mysterious. Seen from the perspective of differential equations (and other viewpoints we’ll see sooner or later) it’s the most natural thing in the world.

Euler’s formula also lets us translate back from trigonometry into exponentials:

$\displaystyle\cos(x)=\frac{\exp(ix)+\exp(-ix)}{2}$
$\displaystyle\sin(x)=\frac{\exp(ix)-\exp(-ix)}{2i}$

And from these formulæ and the differentiation rules for exponentials we can easily work out the differentiation rules for the sine and cosine functions:

$\displaystyle\sin'(x)=\cos(x)$
$\displaystyle\cos'(x)=-\sin(x)$

October 13, 2008 Posted by John Armstrong | Analysis, Calculus, Differential Equations | 21 Comments

First-Degree Homogeneous Linear Equations with Constant Coefficients

Now that we solved one differential equation, let’s try a wider class: “first-degree homogeneous linear equations with constant coefficients”. Let’s break this down.

First-degree: only involves the undetermined function and its first derivative
Homogeneous: harder to nail down, but for our purposes it means that every term involves the undetermined function or its first derivative
Linear: no products of the undetermined function or its first derivative with each other
Constant Coefficients: only multiplying the undetermined function and its derivative by constants

Putting it all together, this means that our equation must look like this:

$\displaystyle a\frac{df}{dx}+bf=0$

We can divide through by $a$ to assume without loss of generality that $a=1$ (if $a=0$ then the equation isn’t very interesting at all).

Now let’s again assume that $f$ is analytic at ${0}$ , so we can write

$\displaystyle f(x)=\sum\limits_{k=0}^\infty a_kx^k$

and

$\displaystyle f'(x)=\sum\limits_{k=0}^\infty (k+1)a_{k+1}x^k$

So our equation reads

$\displaystyle0=\sum\limits_{k=0}^\infty\left((k+1)a_{k+1}+ba_k\right)x^k$

That is, $a_{k+1}=\frac{-b}{k+1}a_k$ for all $k$ , and $a_0=f(0)$ is arbitrary. Just like last time we see that multiplying by $\frac{1}{k+1}$ at each step gives $a_k$ a factor of $\frac{1}{k!}$ . But now we also multiply by $(-b)$ at each step, so we find

$\displaystyle f(x)=\sum\limits_{k=0}^\infty\frac{(-b)^kx^k}{k!}=\sum\limits_{k=0}^\infty\frac{(-bx)^k}{k!}=\exp(-bx)$

And indeed, we can rewrite our equation as $f'(x)=-bf(x)$ . The chain rule clearly shows us that $\exp(-bx)$ satisfies this equation.

In fact, we can immediately see that the function $\exp(kx)$ will satisfy many other equations, like $f''(x)=k^2f(x)$ , $f'''(x)=k^3f(x)$ , and so on.

October 10, 2008 Posted by John Armstrong | Analysis, Calculus, Differential Equations | 3 Comments

The Exponential Differential Equation

So we long ago defined the exponential function $\exp$ to be the inverse of the logarithm, and we showed that it satisfied the exponential property. Now we’ve got another definition, using a power series, which is its Taylor series at ${0}$ . And we’ve shown that this definition also satisfies the exponential property.

But what really makes the exponential function what it is? It’s the fact that the larger the function’s value gets, the faster it grows. That is, the exponential function satisfies the equation $f(x)=f'(x)$ . We already knew this about $\exp$ , but there we ultimately had to use the fact that we defined the logarithm $\ln$ to have a specified derivative. Here we use this property itself as a definition.

This is our first “differential equation”, which relates a function to its derivative(s). And because differentiation works so nicely for power series, we can use them to solve differential equations.

So let’s take our equation as a case in point. First off, any function $f$ that satisfies this equation must by definition be differentiable. And then, since it’s equal to its own derivative, this derivative must itself be differentiable, and so on. So at the very least our function must be infinitely differentiable. Let’s go one step further and just assume that it’s analytic at ${0}$ . Since it’s analytic, we can expand it as a power series.

So we have some function defined by a power series around ${0}$ :

$\displaystyle f(x)=\sum\limits_{k=0}^\infty a_kx^k$

We can easily take the derivative

$\displaystyle f'(x)=\sum\limits_{k=0}^\infty (k+1)a_{k+1}x^k$

Setting these two power series equal, we find that $a_0=1a_1$ , $a_1=2a_2$ , $a_2=3a_3$ , and so on. In general:

$\displaystyle a_k=\frac{1}{k}a_{k-1}=\frac{1}{k}\frac{1}{k-1}a_{k-2}=...=\frac{1}{k}\frac{1}{k-1}...\frac{1}{3}\frac{1}{2}\frac{1}{1}a_0=\frac{a_0}{k!}$

And we have no restriction on $a_0$ . Thus we come up with our series solution

$\displaystyle f(x)=\sum\limits_{k=0}^\infty\frac{a_0}{k!}x^k=a_0\sum\limits_{k=0}^\infty\frac{x^k}{k!}$

which is just $a_0=f(0)$ times the series definition of our exponential function $\exp$ ! If we set the initial value $f(0)=a_0=1$ , then the unique solution to our equation is the function

$\displaystyle\exp(x)=\sum\limits_{k=0}^\infty\frac{x^k}{k!}$

which is our new definition of the exponential function. The differential equation motivates the series, and the series gives us everything else we need.

October 10, 2008 Posted by John Armstrong | Analysis, Calculus, Differential Equations | 3 Comments

The Exponential Series

What is it that makes the exponential what it is? We defined it as the inverse of the logarithm, and this is defined by integrating $\frac{1}{x}$ . But the important thing we immediately showed is that it satisfies the exponential property.

But now we know the Taylor series of the exponential function at ${0}$ :

$\displaystyle\exp(x)=\sum\limits_{k=0}^\infty\frac{x^k}{k!}$

In fact, we can work out the series around any other point the same way. Since all the derivatives are the exponential function back again, we find

$\displaystyle\exp(x)=\sum\limits_{k=0}^\infty\frac{\exp(x_0)}{k!}(x-x_0)^k$

Or we could also write this by expanding around $a$ and writing the relation as a series in the displacement $b=x-a$ :

$\displaystyle\exp(a+b)=\sum\limits_{l=0}^\infty\frac{\exp(a)}{l!}b^l$

Then we can expand out the $\exp(a)$ part as a series itself:

$\displaystyle\exp(a+b)=\sum\limits_{l=0}^\infty\left(\sum\limits_{k=0}^\infty\frac{a^k}{k!}\right)\frac{b^l}{l!}$

But then (with our usual handwaving about rearranging series) we can pull out the inner series since it doesn’t depend on the outer summation variable at all:

$\displaystyle\exp(a+b)=\left(\sum\limits_{k=0}^\infty\frac{a^k}{k!}\right)\left(\sum\limits_{l=0}^\infty\frac{b^l}{l!}\right)$

And these series are just the series defining $\exp(a)$ and $\exp(b)$ , respectively. That is, we have shown the exponential property $\exp(a+b)=\exp(a)\exp(b)$ directly from the series expansion.

That is, whatever function the power series $\sum\limits_{k=0}^\infty\frac{x^k}{k!}$ defines, it satisfies the exponential property. In a sense, the fact that the inverse of this function turns out to be the logarithm is a big coincidence. But it’s a coincidence we’ll tease out tomorrow.

For now I’ll note that this important exponential property follows directly from the series. And we can write down the series anywhere we can add, subtract, multiply, divide (at least by integers), and talk about convergence. That is, the exponential series makes sense in any topological ring of characteristic zero. For example, we can define the exponential of complex numbers by the series

$\displaystyle\exp(z)=\sum\limits_{k=0}^\infty\frac{z^k}{k!}$

Finally, this series will have the exponential property as above, so long as the ring is commutative (like it is for the complex numbers). In more general rings there’s a generalized version of the exponential property, but I’ll leave that until we eventually need to use it.

October 8, 2008 Posted by John Armstrong | Analysis, Calculus, Power Series | 3 Comments

The Taylor Series of the Exponential Function

Sorry for the lack of a post yesterday, but I was really tired after this weekend.

So what functions might we try finding a power series expansion for? Polynomials would be boring, because they already are power series that cut off after a finite number of terms. What other interesting functions do we have?

Well, one that’s particularly nice is the exponential function $\exp$ . We know that this function is its own derivative, and so it has infinitely many derivatives. In particular, $\exp(0)=1$ , $\exp'(0)=1$ , $\exp''(0)=1$ , …, $\exp^{(n)}(0)=1$ , and so on.

So we can construct the Taylor series at ${0}$ . The coefficient formula tells us

$\displaystyle a_k=\frac{\exp^{(k)}(0)}{k!}=\frac{1}{k!}$

which gives us the series

$\displaystyle\sum\limits_{k=0}^\infty\frac{x^k}{k!}$

We use the ratio test to calculate the radius of convergence. We calculate

$\displaystyle\limsup\limits_{k\rightarrow\infty}\left|\frac{\frac{x^{k+1}}{(k+1)!}}{\frac{x^k}{k!}}\right|=\limsup\limits_{k\rightarrow\infty}\left|\frac{x^{k+1}k!}{x^k(k+1)!}\right|=\limsup\limits_{k\rightarrow\infty}\left|\frac{x}{(k+1)}\right|=0$

Thus the series converges absolutely no matter what value we pick for $x$ . The radius of convergence is thus infinite, and the series converges everywhere.

But does this series converge back to the exponential function? Taylor’s Theorem tells us that

$\displaystyle\exp(x)=\left(\sum\limits_{k=0}^n\frac{x^k}{k!}\right)+R_n(x)$

where there is some $\xi_n$ between ${0}$ and $x$ so that $R_n(x)=\frac{\exp(\xi_n)x^n}{(n+1)!}$ .

Now the derivative of $\exp$ is $\exp$ again, and $\exp$ takes only positive values. And so we know that $\exp$ is everywhere increasing. What does this mean? Well, if $x\leq0$ then $\xi_n\leq0$ , and so $\exp(\xi_n)\leq\exp(0)=1$ . On the other hand if $x\geq0$ then $\xi_n\leq nx$ , and so $\exp(\xi_n)\leq\exp(x)$ . Either way, we have some uniform bound $M$ on $\exp(\xi_n)$ no matter what the $\xi_n$ are.

So now we know $R_n(x)\leq\frac{Mx^n}{(n+1)!}$ . And it’s not too hard to see (though I can’t seem to find it in my archives) that $n!$ grows much faster than $x^n$ for any fixed $x$ . Basically, the idea is that each time you’re multiplying by $\frac{x}{n+1}$ , which eventually gets less than and stays less than one. The upshot is that the remainder term $R_n(x)$ must converge to ${0}$ for any fixed $x$ , and so the series indeed converges to the function $\exp(x)$ .

October 7, 2008 Posted by John Armstrong | Analysis, Calculus, Power Series | 5 Comments

Fall Break

WKU is off for a few days, so I’ll take a break too. I should be back Monday.

October 1, 2008 Posted by John Armstrong | Uncategorized | Leave a Comment

Taylor’s Theorem again

What you say? Taylor’s Theorem again? Well, yes. But not quite. Now instead of showing it as an extension of the Fundamental Theorem of Calculus we’ll see it’s an extension of the Differential Mean Value Theorem. And this despite the most strenuous objections of my resident gadfly that the DMVT and FToC have absolutely nothing whatsoever to do with each other, never have, and never will.

Remember that Taylor’s theorem tells us that if $f$ is $n+1$ times continuously differentiable we can write

$\displaystyle f(x)=\left(\sum\limits_{k=0}^n\frac{f^{(k)}(x_0)}{k!}(x-x_0)^k\right)+R_n(x)$

where the remainder term $R_n(x)$ is given in the integral form

$\displaystyle R_n(x)=\int\limits_{x_0}^x\frac{f^{(n+1)}(t)}{n!}(x-t)^ndt$

What I want to say today is that it can also be given in a different form. Specifically, there is some $\xi$ between $x_0$ and $x$ so that

$\displaystyle R_n(x)=\frac{f^{(n+1)}(\xi)}{(n+1)!}(x-x_0)^{n+1}$

Now, we could actually weaken the assumptions on $f$ slightly and work up a whole new induction from the ground up using a souped-up version of the DMVT. In particular, notice that the $n=0$ case says there is a $\xi$ between $x_0$ and $x$ so that

$\displaystyle f(x)=f(x_0)+f'(\xi)(x-x_0)$

which is just a different way of writing the DMVT itself.

Instead of doing all that hard work, I’ll prove that this form of the remainder term holds under the same conditions as the integral version. And I’ll do this by using a form of the Integral Mean Value Theorem for Riemann-Stieltjes integration.

Specifically, we’ve got an integral to evaluate

$\displaystyle\int\limits_{x_0}^xf^{(n+1)}(t)\frac{(x-t)^n}{n!}dt$

We set

$\displaystyle d\alpha=\frac{(x-t)^n}{n!}dt$

and antidifferentiate to find

$\displaystyle\alpha=-\frac{(x-t)^{n+1}}{(n+1)!}$

Now the IMVT tells us that there is a $\xi$ between $x_0$ and $x$ so that

$\displaystyle\begin{aligned}f^{n+1}(\xi)=\frac{1}{-\frac{(x-x)^{n+1}}{(n+1)!}+\frac{(x-x_0)^{n+1}}{(n+1)!}}\int\limits_{x_0}^xf^{(n+1)}(t)\frac{(x-t)^n}{n!}dt\\\frac{(n+1)!}{(x-x_0)^{n+1}}\int\limits_{x_0}^xf^{(n+1)}(t)\frac{(x-t)^n}{n!}dt\end{aligned}$

That is

$\displaystyle\frac{f^{(n+1)}(\xi)}{(n+1)!}(x-x_0)^{n+1}=\int\limits_{x_0}^xf^{(n+1)}(t)\frac{(x-t)^n}{n!}dt=R_n(x)$

Again, if we can make $R_n(x)$ converge to ${0}$ in a neighborhood of $x_0$ , then the infinitely-differentiable function $f$ is actually analytic. Here the limit gets fidgety because we don’t know where between $x_0$ and $x$ we might pick $\xi$ , and the $\xi$ jumps around as we use larger and larger $n$ . But if we can keep good control on the size of all of the derivatives of $f$ near $x_0$ , this can be a useful strategy.

October 1, 2008 Posted by John Armstrong | Analysis, Calculus | 4 Comments

Next Entries »

About this weblog

This is mainly an expository blath, with occasional high-level excursions, humorous observations, rants, and musings. The main-line exposition should be accessible to the “Generally Interested Lay Audience”, as long as you trace the links back towards the basics. Check the sidebar for specific topics (under “Categories”).

I’m in the process of tweaking some aspects of the site to make it easier to refer back to older topics, so try to make the best of it for now.

RSS Feeds

RSS - Posts
RSS - Comments
Feedback

Got something to say? Anonymous questions, comments, and suggestions at Formspring.me!
Subjects
Archives

October 2008

M T W T F S S

« Sep Nov »

1 2 3 4 5

6 7 8 9 10 11 12

13 14 15 16 17 18 19

20 21 22 23 24 25 26

27 28 29 30 31
Search for:

The Unapologetic Mathematician

Mathematics for the interested outsider

Sine and Cosine

First-Degree Homogeneous Linear Equations with Constant Coefficients

The Exponential Differential Equation

The Exponential Series

The Taylor Series of the Exponential Function

Fall Break

Taylor’s Theorem again

About this weblog

Recent Posts

Blogroll

Art

Astronomy

Computer Science

Education

Mathematics

Me

Philosophy

Physics

Politics

Science

RSS Feeds

Feedback

Subjects

Archives

Site info

Follow “The Unapologetic Mathematician”

M	T	W	T	F	S	S
« Sep				Nov »
		1	2	3	4	5
6	7	8	9	10	11	12
13	14	15	16	17	18	19
20	21	22	23	24	25	26
27	28	29	30	31