In its simplest form, a flag is simply a strictly-increasing sequence of subspaces of a given finite-dimensional vector space. And we almost always say that a flag starts with and ends with . In the middle we have some other subspaces, each one strictly including the one below it. We say that a flag is “complete” if — and thus — and for our current purposes all flags will be complete unless otherwise mentioned.
The useful thing about flags is that they’re a little more general and “geometric” than ordered bases. Indeed, given an ordered basis we have a flag on : define to be the span of . As a partial converse, given any (complete) flag we can come up with a not-at-all-unique basis: at each step let be the preimage in of some nonzero vector in the one-dimensional space .
We say that an endomorphism of “stabilizes” a flag if it sends each back into itself. In fact, we saw something like this in the proof of Lie’s theorem: we build a complete flag on the subspace , building the subspace up one basis element at a time, and then showed that each stabilized that flag. More generally, we say a collection of endomorphisms stabilizes a flag if all the endomorphisms in the collection do.
So, what do Lie’s and Engel’s theorems tell us about flags? Well, Lie’s theorem tells us that if is solvable then it stabilizes some flag in . Equivalently, there is some basis with respect to which the matrices of all elements of are upper-triangular. In other words, is isomorphic to some subalgebra of . We see that not only is solvable, it is in a sense the archetypal solvable Lie algebra.
The proof is straightforward: Lie’s theorem tells us that has a common eigenvector . We let this span the one-dimensional subspace and consider the action of on the quotient . Since we know that the image of in will again be solvable, we get a common eigenvector . Choosing a pre-image with we get our second basis vector. We can continue like this, building up a basis of such that at each step we can write for all and some .
For nilpotent , the same is true — of course, nilpotent Lie algebras are automatically solvable — but Engel’s theorem tells us more: the functional $\lambda$ must be zero, and the diagonal entries of the above matrices are all zero. We conclude that any nilpotent is isomorphic to some subalgebra of . That is, not only is nilpotent, it is the archetype of all nilpotent Lie algebras in just the same way as is the archetypal solvable Lie algebra.
More generally, if is any solvable (nilpotent) Lie algebra and is any finite-dimensional representation of , then we know that the image is a solvable (nilpotent) linear Lie algebra acting on , and thus it must stabilize some flag of . As a particular example, consider the adjoint action ; a subspace of invariant under the adjoint action of is just the same thing as an ideal of , so we find that there must be some chain of ideals:
where . Given such a chain, we can of course find a basis of with respect to which the matrices of the adjoint action are all in ().
In either case, we find that is nilpotent. Indeed, if is already nilpotent this is trivial. But if is merely solvable, we see that the matrices of the commutators for lie in
But since is a homomorphism, this is the matrix of acting on , and obviously its action on the subalgebra is nilpotent as well. Thus each element of is ad-nilpotent, and Engel’s theorem then tells us that is a nilpotent Lie algebra.
The lemma leading to Engel’s theorem boils down to the assertion that there is some common eigenvector for all the endomorphisms in a nilpotent linear Lie algebra on a finite-dimensional nonzero vector space . Lie’s theorem says that the same is true of solvable linear Lie algebras. Of course, in the nilpotent case the only possible eigenvalue was zero, so we may find things a little more complicated now. We will, however, have to assume that is algebraically closed and that no multiple of the unit in is zero.
We will proceed by induction on the dimension of using the same four basic steps as in the lemma: find an ideal of codimension one, so we can write for some ; find common eigenvectors for ; find a subspace of such common eigenvectors stabilized by ; find in that space an eigenvector for .
First, solvability says that properly includes , or else the derived series wouldn’t be able to even start heading towards . The quotient must be abelian, with all brackets zero, so we can pick any subspace of this quotient with codimension one and it will be an ideal. The preimage of this subspace under the quotient projection will then be an ideal of codimension one.
Now, is a subalgebra of , so we know it’s also solvable, so induction tells us that there’s a common eigenvector for the action of . If is zero, then must be one-dimensional abelian, in which case the proof is obvious. Otherwise there is some linear functional defined by
Of course, is not the only such eigenvector; we define the (nonzero) subspace by
Next we must show that sends back into itself. To see this, pick and and check that
But if , then we’d have ; we need to verify that . In the nilpotent case — Engel’s theorem — the functional was constantly zero, so this was easy, but it’s a bit harder here.
Fixing and we pick to be the first index where the collection is linearly independent — the first one where we can express as the linear combination of all the previous . If we write for the subspace spanned by the first of these vectors, then the dimension of grows one-by-one until we get to , and from then on.
I say that each of the are invariant under each . Indeed, we can prove the congruence
that is, acts on by multiplication by , plus some “lower-order terms”. For this is the definition of ; in general we have
for some .
And so we conclude that, using the obvious basis of the action of on this subspace is in the form of an upper-triangular matrix with down the diagonal. The trace of this matrix is . And in particular, the trace of the action of on is . But and both act as endomorphisms of — the one by design and the other by the above proof — and the trace of any commutator is zero! Since must have an inverse we conclude that .
Okay so that checks out that the action of sends back into itself. We finish up by picking some eigenvector of , which we know must exist because we’re working over an algebraically closed field. Incidentally, we can then extend to all of by using .