Linear Equations

Okay, now I really should introduce one of the most popular applications of linear algebra, at least outside mathematics. Matrices can encode systems of linear equations, and matrix algebra can be used to solve them.

What is a linear equation? It’s simply an algebraic equation where each variable shows up at most to the first power. For example, we could consider the equation

$3x=12$

and clearly we can solve this equation by dividing by $3$ on each side to find $x=4$ . Of course, there could be more than one variable.

Sometimes people might use different names like $x$ , $y$ , and $z$ , but since we want to be open-ended about things we’ll just say $x^1$ , $x^2$ , $x^3$ , and so on. Notice here that because variables can only show up to the first power, there is no ambiguity about writing our indices as superscripts — something we’ve done before. Anyhow, we might write an equation

$3x^1+4x^2=12$

Now we have many different possible solutions. We could set $x^1=4$ and $x^2=0$ , or we could set $x^1=0$ and $x^2=3$ , or all sorts of combinations in between. This one equation is not enough to specify a unique solution.

Things might change, though, if we add more equations. Consider the system

$3x^1+4x^2=12$
$x^1-x^2=11$

Now we can rewrite the second equation as $x^1=11+x^2$ , and then drop this into the first equation to find $3(11+x^2)+4x^2=12$ , or $33+7x^2=12$ . This just involves the one variable, and we can easily solve it to find $x^2=-3$ , which quickly yields $x^1=8$ . We now have a single solution for the pair of equations.

What if we add another equation? Consider

$3x^1+4x^2=12$
$x^1-x^2=11$
$x^1+x^2=10$

Now we know that the only values solving both of the first two equations are $x^1=8$ and $x^2=-3$ . But then $x^1+x^2=5$ , so the third equation cannot possibly be satisfied. Too many equations can be impossible to solve.

In general, things work out when we’ve got one equation for each variable. As we move forwards we’ll see how to express this fact more concretely in terms of matrices and vector spaces.

19 Comments »

Interesting, i have authors use subscripts. . . . but never seen superscript. It seems to me that this could lead to some ambiguity.

Comment by Ryan Smith | July 3, 2008 | Reply
Well, I’m using superscripts so it will mesh with the summation convention when I write out a linear system as a matrix equation. Then the many variables will be the components of a single vector variable.

Comment by John Armstrong | July 3, 2008 | Reply
so ambiguous… maybe you want to think of some better notation…

Comment by Mgccl | July 4, 2008 | Reply
Mgccl: I’m not just blindly following convention here. I specifically chose superscript indices for the very specific reason I just pointed out.

Comment by John Armstrong | July 4, 2008 | Reply
It might be useful to specify the domain and range. Are these variable real or complex, for example? It is important to specify that they are not restricted to integers, otherwise we get the undecidability of Diophantine equations, via Godel numbers, Hilbert’s tenth problem, and Yuri Matiyasevich.

Comment by Jonathan Vos Post | July 4, 2008 | Reply
JVP: There’s no “domain” and “range” yet. You’re jumping ahead to the next step, where we encode a linear system in a matrix, and thus as a linear map.

Comment by John Armstrong | July 4, 2008 | Reply
Even taken over the integers, I don’t think undecidability issues come into play here; these are just linear systems we’re talking about.

Comment by Todd Trimble | July 4, 2008 | Reply
how about superscript before the variable, instead of after?

3 ^1x+4 ^2x = 12
^1x -^2x = 11

Comment by emf | July 8, 2008 | Reply
Expanding on Todd Trimble’s point, remembering the decidability of Presburger arithmetic should obviate any decidability concerns for solving linear equations in the integral context (and similarly by Tarski’s results for the real and complex contexts). But the superscript notation has perhaps fed confusion over just what kinds of equations are being looked at.

Comment by Sridhar Ramesh | July 8, 2008 | Reply
[…] consider a system of linear equations. We’ll use the variables , , and so on up to ; and we’ll let there be equations. […]

Pingback by The Matrix of a Linear System « The Unapologetic Mathematician | July 11, 2008 | Reply
[…] Linear Systems An important special case of a linear system is a set of homogenous equations. All this means (in this case) is that the right side of each of […]

Pingback by Homogenous Linear Systems « The Unapologetic Mathematician | July 14, 2008 | Reply
[…] must there always be a particular solution to begin with? Clearly not. When we first talked about linear systems we mentioned the […]

Pingback by Unsolvable Inhomogenous Systems « The Unapologetic Mathematician | July 18, 2008 | Reply
[…] Today I want to talk about the index of a linear map , which I’ll motivate in terms of a linear system for some linear map […]

Pingback by The Index of a Linear Map « The Unapologetic Mathematician | July 22, 2008 | Reply
Hi,
Any thoughts on how to find the range for each independent variable, for a given range of dependent variable in a linear equation with, say 1 dependent and 3 independent variables? i’m struggling with this, despite trying Monte-Carlo simulation…

Comment by srinivas | February 18, 2009 | Reply
I’m not sure what you mean, srinivas. Can you give a more explicit example?

Comment by John Armstrong | February 18, 2009 | Reply
Hi John,
Thanks for the response. What I was referring to is a multiple regression eqation (y = ax1 + bx2 + cx3 + d). In this case for a given range of y, there exist a whole lot of values for each of x1, x2 and x3. What i’m interested in knowing is whether I can come up with the range for one variable, say x3 which I believe has highest impact on y. All this while not keeping x1 or x2 as constant.

Please tell me if it is possible. (I looked at simulation to give me an answer, but was unsuccessful)

Comment by srinivas | February 19, 2009 | Reply
I still don’t know what you mean by “has the highest impact on $y$ “. The point of linear functions is that the effect on the output of a change in the input is constant over the entire domain.

Comment by John Armstrong | February 19, 2009 | Reply
[…] SVD also comes in handy for solving systems of linear equations. Let’s say we have a system written down as the matrix […]

Pingback by The Meaning of the SVD « The Unapologetic Mathematician | August 18, 2009 | Reply
[…] The row echelon form of a matrix isn’t unique, but it’s still useful for solving systems of linear equations. If we have a system written in matrix form […]

Pingback by Solving Equations with Gaussian Elimination « The Unapologetic Mathematician | September 2, 2009 | Reply

About this weblog

This is mainly an expository blath, with occasional high-level excursions, humorous observations, rants, and musings. The main-line exposition should be accessible to the “Generally Interested Lay Audience”, as long as you trace the links back towards the basics. Check the sidebar for specific topics (under “Categories”).

I’m in the process of tweaking some aspects of the site to make it easier to refer back to older topics, so try to make the best of it for now.

Recent Posts
Blogroll
Art
- Sketches of Topology
Astronomy
- Bad Astronomy
- Starts With a Bang
Computer Science
Education
Mathematics
Me
- DrMathochist
- The Unapologetic Programmer
Philosophy
Physics
Politics
Science

The Unapologetic Mathematician

Mathematics for the interested outsider