Matrix notation
I just spent all day on the road back to NOLA to handle some end-of-month business, clean out my office, and so on. This one will have to do for today and tomorrow.
It gets annoying to write out matrices using the embedded LaTeX here, but I suppose I really should, just for thoroughness’ sake.
In general, a matrix is a collection of field elements with an upper and a lower index. We can write out all these elements in a rectangular array. The upper index should list the rows of our array, while the lower index should list the columns. The matrix with entries
for
running from
to
and
running from
to
is written out in full as
We call this an matrix, because the array is
rows high and
columns wide.
There is a natural isomorphism . This means that every vector in dimension
, written out in the components relative to a given basis, can be seen as an
“column vector”:
Similarly, a linear functional on an -dimensional space can be written as a
“row vector”:
Notice that evaluation of linear transformations is now just a special case of matrix multiplication! Let’s practice by writing out the composition of a linear functional , a linear map
, and a vector
.
A matrix product makes sense if and only if the number of columns in the left-hand matrix is the same as the number of rows in the right-hand matrix. That is, an and an
can be multiplied. The result will be an
matrix. We calculate it by taking a row from the left-hand matrix and a column from the right-hand matrix. Since these are the same length (by assumption) we can multiply corresponding elements and sum up.
In the example above, the matrix
and the
matrix
can be multiplied. There is only one column in the latter to pick, so we simply choose row
out of
on the left:
. Multiplying corresponding elements and summing gives the single field element
(remember the summation convention). We get
of these elements — one for each row — and we arrange them in a new
matrix:
Then we can multiply the row vector by this column vector to get the
matrix:
Just like we slip back and forth between vectors and matrices, we will usually consider a field element and the
matrix with that single entry as being pretty much the same thing.
The first multiplication here turned an -dimensional (column) vector into an
-dimensional one, reflecting the source and target of the transformation
. Then we evaluated the linear functional
on the resulting vector. But by the associativity of matrix multiplication we could have first multiplied on the left:
turning the linear functional on into one on
. But this is just the dual transformation
! Then we can evaluate this on the column vector to get the same result:
.
There is one slightly touchy thing we need to be careful about: Kronecker products. When the upper index is a pair with
and
we have to pick an order on the set of such pairs. We’ll always use the “lexicographic” order. That is, we start with
, then
, and so on until
before starting over with
,
, and so on. Let’s write out a couple examples just to be clear:
So the Kronecker product depends on the order of multiplication. But this dependence is somewhat illusory. The only real difference is reordering the bases we use for the tensor products of the vector spaces involved, and so a change of basis can turn one into the other. This is an example of how matrices can carry artifacts of our choice of bases.
Matrices IV
Like we saw with the tensor product of vector spaces, the dual space construction turns out to be a functor. In fact, it’s a contravariant functor. That is, if we have a linear transformation we get a linear transformation
. As usual, we ask what this looks like for matrices.
First, how do we define the dual transformation? It turns out this is the contravariant functor represented by . That is, if
is a linear functional, we define
. In terms of the action on vectors,
Now let’s assume that and
are finite-dimensional, and pick bases
and
for
and
, respectively. Then the linear transformation
has matrix coefficients
. We also get the dual bases
of
and
of
.
Given a basic linear functional on
, we want to write
in terms of the
. So let’s evaluate it on a generic basis vector
and see what we get. The formula above shows us that
In other words, we can write . The same matrix works, but we use its indices differently.
In general, given a linear functional with coefficients
we find the coefficients of
as
. The value
becomes
. Notice that the summation convention tells us this must be a scalar (as we expect) because there are no unpaired indices. Also notice that because we can use the same matrix for two different transformations we seem to have an ambiguity: is the lower index running over a basis for
or one for
? Luckily, since every basis gives rise to a dual basis, we don’t need to care. Both spaces have the same dimension anyhow.
Dual Spaces
Another thing vector spaces come with is duals. That is, given a vector space we have the dual vector space
of “linear functionals” on
— linear functions from
to the base field
. Again we ask how this looks in terms of bases.
So let’s take a finite-dimensional vector space with basis
, and consider some linear functional
. Like any linear function, we can write down matrix coefficients
. Notice that since our target space (the base field
) is only one-dimensional, we don’t need another index to count its basis.
Now let’s consider a specially-crafted linear functional. We can define one however we like on the basis vectors and then let linearity handle the rest. So let’s say our functional takes the value
on
and the value
on every other basis element. We’ll call this linear functional
. Notice that on any vector we have
so it returns the coefficient of . There’s nothing special about
here, though. We can define functionals
by setting
. This is the “Kronecker delta”, and it has the value
when its two indices match, and
when they don’t.
Now given a linear functional with matrix coefficients
, let’s write out a new linear functional
. What does this do to basis elements?
so this new transformation has exactly the same matrix as does. It must be the same transformation! So any linear functional can be written uniquely as a linear combination of the
, and thus they form a basis for the dual space. We call
the “dual basis” to
.
Now if we take a generic linear functional and evaluate it on a generic vector
we find
Once we pick a basis for we immediately get a basis for
, and evaluation of a linear functional on a vector looks neat in terms of these bases.
Matrices III
Given two finite-dimensional vector spaces and
, with bases
and
respectively, we know how to build a tensor product: use the basis
.
But an important thing about the tensor product is that it’s a functor. That is, if we have linear transformations and
, then we get a linear transformation
. So what does this operation look like in terms of matrices?
First we have to remember exactly how we get the tensor product . Clearly we can consider the function
. Then we can compose with the bilinear function
to get a bilinear function from
to
. By the universal property, this must factor uniquely through a linear function
. It is this map we call
.
We have to pick bases of
and
of
. This gives us a matrix coefficients
for
and
for
. To calculate the matrix for
we have to evaluate it on the basis elements
of
. By definition we find:
that is, the matrix coefficient between the index pair and the index pair
is
.
It’s not often taught anymore, but there is a name for this operation: the Kronecker product. If we write the matrices (as opposed to just their coefficients) and
, then we write the Kronecker product
.
Sunday Samples 70
Today some people I know are getting together and heading out to a firing range out in the boondocks of Maryland. And so I think I’ll pull out Moby’s cover of “That’s When I Reach For My Revolver”. Yes, I know the original was by Mission of Burma, so if you want a more political spin today you can go listen to that version.
Read more »
Go Speed Go!
I know I don’t usually do reviews here, but I trekked out to the Udvar-Hazy annex of the Air & Space Museum last night (all the way in Chantilly) to see Speed Racer on IMAX. Go see it, but only if you can find it near you in IMAX. It’s just not worth it for a regular screen.
The quote of the night:
“I should have worn my glasses. All I saw was bright colors, blurs, and I couldn’t tell what was going on.”
“No, that’s what it was supposed to look like.”
Tensor Products and Bases
Since we’re looking at vector spaces, which are special kinds of modules, we know that has a tensor product structure. Let’s see what this means when we pick bases.
First off, let’s remember what the tensor product of two vector spaces and
is. It’s a new vector space
and a bilinear (linear in each of two variables separately) function
satisfying a certain universal property. Specifically, if
is any bilinear function it must factor uniquely through
as
. The catch here is that when we say “linear” and “bilinear” we mean that the functions preserve both addition and scalar multiplication. As with any other universal property, such a tensor product will be uniquely defined up to isomorphism.
So let’s take finite-dimensional vector spaces and
, and bases
of
and
of
. I say that the vector space with basis
, and with the bilinear function
is a tensor product. Here the expression
is just a name for a basis element of the new vector space. Such elements are indexed by the set of pairs
, where
indexes a basis for
and
indexes a basis for
.
First off, what do I mean by the bilinear function ? Just as for linear functions, we can define bilinear functions by defining them on bases. That is, if we have
and
, we get the vector
in our new vector space, with coefficients .
So let’s take a bilinear function and define a linear function
by setting
We can easily check that does indeed factor as desired, since
so on basis elements. By linearity, they must agree for all pairs
. It should also be clear that we can’t define
any other way and hope to satisfy this equation, so the factorization is unique.
Thus if we have bases of
and
of
, we immediately get a basis
of
. As a side note, we immediately see that the dimension of the tensor product of two vector spaces is the product of their dimensions.
Accomplishment?
To show that my underemployment isn’t wasted:

This used to be a tree.
Insert your own joke about runaway pruning algorithms here.
Matrices II
With the summation convention firmly in hand, we continue our discussion of matrices.
We’ve said before that the category of vector space is enriched over itself. That is, if we have vector spaces and
over the field
, the set of linear transformations
is itself a vector space over
. In fact, it inherits this structure from the one on
. We define the sum and the scalar product
for linear transformations and
from
to
, and for a constant
. Verifying that these are also linear transformations is straightforward.
So what do these structures look like in the language of matrices? If and
are finite-dimensional, let’s pick bases
of
and
of
. Now we get matrix coefficients
and
, where
indexes the basis of
and
indexes the basis of
. Now we can calculate the matrices of the sum and scalar product above.
We do this, as usual, by calculating the value the transformations take at each basis element. First, the sum:
and now the scalar product:
so we calculate the matrix coefficients of the sum of two linear transformations by adding the corresponding matrix coefficients of each transformation, and the matrix coefficients of the scalar product by multiplying each coefficient by the same scalar.
About this weblog
This is mainly an expository blath, with occasional high-level excursions, humorous observations, rants, and musings. The main-line exposition should be accessible to the “Generally Interested Lay Audience”, as long as you trace the links back towards the basics. Check the sidebar for specific topics (under “Categories”).
I’m in the process of tweaking some aspects of the site to make it easier to refer back to older topics, so try to make the best of it for now.

