\section{ Eigenvalues and Eigenvectors II }

Last time, we developed the idea of eigenvalues and eigenvectors in the case of linear transformations $\Re^2\rightarrow \Re^2$.  In this section, we will develop the idea more generally.

\begin{definition}
For a linear transformation $L:V\rightarrow V$, then $\lambda$ is an \emph{eigenvalue}\index{Eigenvalue} of $L$ with \emph{eigenvector}\index{Eigenvector} $v\neq 0_V$ if

\[
Lv=\lambda v.
\]  
\end{definition}
This equation says that the direction of $v$ is invariant (unchanged) under~$L$.  

Let's try to understand this equation better in terms of matrices.
Let $V$ be a finite-dimensional vector space (we'll explain what it means to be finite-dimensional in more detail later; for now, take this to mean $\Re^n$), and let $L:V\rightarrow V$.  

\begin{remark}[Matrix of a Linear Transformation]\index{Matrix!of a linear transformation}
Any vector in $\Re^n$ can be written as a linear combination of the \emph{standard basis vectors}\index{Standard basis} $\{e_i | i \in \{1, \ldots, n\} \}$.  The vector $e_i$ has a one in the $i$th position, and zeros everywhere else.  {\it I.e.}
$$
e_1=\colvec{1\\ 0\\ \vdots \\0}\, ,\quad e_2=\colvec{0\\ 1\\ \vdots \\0}\, ,\cdots\quad e_n=\colvec{0\\ 0\\ \vdots \\1 }\, .
$$
Then to find the matrix of any linear transformation $L:\Re^n \rightarrow \Re^n$, it suffices to know what $L(e_i)$ is for every $i$.  

For any matrix $M$, observe that $Me_i$ is equal to the $i$th column of $M$.  Then if the $i$th column of $M$ equals $L(e_i)$ for every $i$, then $Mv=L(v)$ for every $v\in \Re^n$.  Then the matrix representing $L$ in the standard basis is just the matrix whose $i$th column is $L(e_i)$.
\end{remark}

Since we can represent $L$ by a square matrix $M$, we find eigenvalues $\lambda$ and associated eigenvectors $v$ by solving the homogeneous system

\[
(M-\lambda I)v=0.
\]
This system has non-zero solutions if and only if the matrix

\[
M-\lambda I
\]
is singular, and so we require that

\[
\det (\lambda I-M) = 0.
\]

The left hand side of this equation is a polynomial in the variable $\lambda$ called the \emph{characteristic polynomial}\index{Characteristic polynomial} $P_M(\lambda)$ of $M$.  For an $n\times n$ matrix, the characteristic polynomial has degree $n$.  Then 
\[
P_M(\lambda) = \lambda^n+c_1\lambda^{n-1}+\ldots+c_n.
\]
Notice that $P_M(0)=\det (-M)=(-1)^n\det M$.

The \emph{fundamental theorem of algebra}\index{Fundamental theorem of algebra} states that any polynomial can be factored into a product of linear terms over $\C$.  Then there exists a collection of $n$ complex numbers $\lambda_i$ (possibly with repetition) such that
\[
P_M(\lambda)=(\lambda-\lambda_1)(\lambda-\lambda_2)\ldots(\lambda-\lambda_n), \qquad P_M(\lambda_i)=0
\]
The eigenvalues $\lambda_i$ of $M$ are exactly the roots of $P_M(\lambda)$.  These eigenvalues could be real or complex or zero, and they need not all be different.  The number of times that any given root $\lambda_i$ appears in the collection of eigenvalues is called its \emph{multiplicity}\index{Eigenvalue!multiplicity of}.




\begin{example}
Let $L$ be the linear transformation $L:\Re^3\rightarrow \Re^3$ given by $$L(x,y,z)=(2x+y-z, x+2y-z, -x-y+2z)\, .$$ The matrix $M$ representing $L$ has columns $Le_i$ for each $i$, so:

\[
\colvec{x\\y\\z} \stackrel{L}{\mapsto} 
\begin{pmatrix}
2 & 1 & -1\\
1 & 2 & -1\\
-1 & -1 & 2\\
\end{pmatrix}\colvec{x\\y\\z}.
\]
Then the characteristic polynomial of $L$ is\footnote{It is often easier (and equivalent) to solve $\det (M-\lambda I)=0$.}
\begin{eqnarray*}
P_M(\lambda)&=&\det \begin{pmatrix}
\lambda - 2 & -1 & 1\\
-1 & \lambda - 2 & 1\\
1 & 1 & \lambda - 2\\
\end{pmatrix}\\[2mm]
&=& (\lambda - 2)[ (\lambda - 2)^2-1 ] + 
[-(\lambda - 2)-1 ] +
[-(\lambda - 2)-1] \\[2mm]
&=& (\lambda - 1)^2(\lambda - 4)
\end{eqnarray*}
Then $L$ has eigenvalues $\lambda_1=1$ (with multiplicity $2$), and $\lambda_2=4$ (with multiplicity $1$).

To find the eigenvectors associated to each eigenvalue, we solve the homogeneous system $(M-\lambda_iI)X=0$ for each $i$.

\begin{itemize}
\item[\underline{$\lambda=4$:}] We set up the augmented matrix for the linear system:
\begin{eqnarray*}
\begin{amatrix}{3}
-2 & 1 &-1 & 0 \\
1 & -2 &-1 & 0 \\
-1 & -1 &-2 & 0 \\
\end{amatrix} 
 & \sim & \begin{amatrix}{3}
1 & -2 &-1 & 0 \\
0 & -3 &-3 & 0 \\
0 & -3 &-3 & 0 \\
\end{amatrix} \\
 & \sim & \begin{amatrix}{3}
1 & 0 & 1 & 0 \\
0 & 1 & 1 & 0 \\
0 & 0 & 0 & 0 \\
\end{amatrix}. \\
\end{eqnarray*}
So we see that $z=t$, $y=-t$, and $x=-t$ gives a formula for eigenvectors in terms of the free parameter $t$.  Any such eigenvector is of the form $t\colvec{-1\\-1\\1}$; thus $L$ leaves a line through the origin invariant.

\item[\underline{$\lambda=1$:}]  Again we set up an augmented matrix and find the solution set:
\begin{eqnarray*}
\begin{amatrix}{3}
1 & 1 &-1 & 0 \\
1 & 1 &-1 & 0 \\
-1 & -1 & 1 & 0 \\
\end{amatrix} 
 & \sim & \begin{amatrix}{3}
1 & 1 &-1 & 0 \\
0 & 0 &0 & 0 \\
0 & 0 &0 & 0 \\
\end{amatrix}.
\end{eqnarray*}
Then the solution set has two free parameters, $s$ and $t$, such that $z=t$, $y=s$, and $x=-s+t$.  Then $L$ leaves invariant the set:
\[
\{ s\colvec{-1\\1\\0}+t\colvec{1\\0\\1} | s,t\in \Re   \}.
\]
This set is a plane through the origin.  So the multiplicity two eigenvalue has two independent eigenvectors, $\colvec{-1\\1\\0}$ and $\colvec{1\\0\\1}$ that determine an invariant plane.
\end{itemize}
\end{example}

\begin{example}
Let $V$ be the vector space of smooth (\textit{i.e.} infinitely differentiable) functions $f:\Re\rightarrow \Re$.  Then the derivative is a linear operator $\frac{d}{dx}:V\rightarrow V$.  What are the eigenvectors of the derivative?  In this case, we don't have a matrix to work with, so we have to make do.

A function $f$ is an eigenvector of $\frac{d}{dx}$ if there exists some number $\lambda$ such that $\frac{d}{dx}f=\lambda f$.  An obvious candidate is the exponential function, $e^{\lambda x}$; indeed, $\frac{d}{dx} e^{\lambda x} = \lambda e^{\lambda x}$.

As such, the operator $\frac{d}{dx}$ has an eigenvector $e^{\lambda x}$ for every $\lambda \in \Re$.

This is actually the whole collection of eigenvectors for $\frac{d}{dx}$; this can be proved using the fact that every infinitely differentiable function has a Taylor series with infinite radius of convergence, and then using the Taylor series to show that if two functions are eigenvectors of $\frac{\partial}{\partial x}$ with eigenvalues $\lambda$, then they are scalar multiples of each other.
\end{example}


\subsection{Eigenspaces}
In the previous example, we found two eigenvectors  $\colvec{-1\\1\\0}$ and $\colvec{1\\0\\1}$ for $L$ with eigenvalue $1$.  Notice that  $\colvec{-1\\1\\0} + \colvec{1\\0\\1}=
\colvec{0\\1\\1}$ is also an eigenvector of $L$ with eigenvalue $1$.  In fact, any linear combination $r\colvec{-1\\1\\0} + s\colvec{1\\0\\1}$ of these two eigenvectors will be another eigenvector with the same eigenvalue.  

More generally, let $\{ v_1, v_2, \ldots \}$ be eigenvectors of some linear transformation $L$ with the same eigenvalue $\lambda$.  A \emph{linear combination}\index{Linear combination} of the $v_i$ can be written $c_1v_1+c_2v_2+\ldots$ for some constants $\{c_1, c_2,\ldots \}$.  Then:
\begin{eqnarray*}
L(c_1v_1+c_2v_2+\ldots) &=& c_1Lv_1+c_2Lv_2+\ldots \text{ by linearity of $L$}\\
&=& c_1\lambda v_1+c_2\lambda v_2+\ldots \text{ since $Lv_i=\lambda v_i$ }\\
&=& \lambda (c_1v_1+c_2v_2+\ldots).
\end{eqnarray*}
So every linear combination of the $v_i$ is an eigenvector of $L$ with the same eigenvalue $\lambda$.
In simple terms, any sum of eigenvectors is again an eigenvector {\it if they share the same eigenvalue}.

The space of all vectors with eigenvalue $\lambda$ is called an \emph{eigenspace}\index{Eigenspace}.  It is, in fact, a vector space contained within the larger vector space $V$:  It contains $0_V$, since $L0_V=0_V=\lambda 0_V$, and is closed under addition and scalar multiplication by the above calculation.  All other vector space properties are inherited from the fact that $V$ itself is a vector space.

An eigenspace is an example of a \emph{subspace}\index{Subspace!notion of} of $V$, a notion that we will explore further next time.



\section*{References}
\begin{itemize}
\item Hefferon, Chapter Three, Section III.1: Representing Linear Maps with Matrices
\item Hefferon, Chapter Five, Section II.3: Eigenvalues and Eigenvectors
\end{itemize}

Wikipedia:
\begin{itemize}
\item \href{http://en.wikipedia.org/wiki/Eigenvalue,_eigenvector_and_eigenspace}{Eigen*}
\item \href{http://en.wikipedia.org/wiki/Characteristic_polynomial}{Characteristic Polynomial}
\item \href{http://en.wikipedia.org/wiki/Linear_map}{Linear Transformations (and matrices thereof)}
\end{itemize}










\section*{Review Questions}

\begin{enumerate}
\item Explain why the characteristic polynomial of an $n\times n$ matrix has degree $n$.  Make your explanation easy to read by starting with some simple examples, and then use properties of the determinant to give a \emph{general} explanation.

\item Compute the characteristic polynomial $P_M(\lambda)$ of the matrix $M=\begin{pmatrix}
a & b \\
c & d
\end{pmatrix}$.  Now, since we can evaluate polynomials on square matrices, we can plug $M$ into its characteristic polynomial and find the {\it matrix} $P_M(M)$.  What do you find from this computation?
Investigate whether something similar holds for $n\times n$ matrices.

\end{enumerate}



\newpage
