\section{Eigenvalues and  Eigenvectors }

\begin{remark}[Matrix of a Linear Transformation]\index{Matrix!of a linear transformation}

Consider a linear transformation $$L: \Re^2\rightarrow \Re^2\, .$$  Suppose we know that $L\colvec{1\\0}=\colvec{a\\c}$ and $L\colvec{0\\1}=\colvec{b\\d}$.  Then, because of linearity, we can determine what $L$ does to any vector $\colvec{x\\y}$:

\[
L\colvec{x\\y}=L(x\colvec{1\\0}+y\colvec{0\\1})=xL\colvec{1\\0}+yL\colvec{0\\1}=x\colvec{a\\c}+y\colvec{b\\d}=\colvec{ax+by\\cx+dy}.
\]
Now notice that for any vector $\colvec{x\\y}$, we have 

\[
\begin{pmatrix}
a & b \\
c & d \\
\end{pmatrix}
\colvec{x\\y}=\colvec{ax+by\\cx+dy}=L\colvec{x\\y}.
\]
Then the matrix $\begin{pmatrix}
a & b \\
c & d \\
\end{pmatrix}$ acts by matrix multiplication in the same way that~$L$ does.  Call this matrix the \emph{matrix of $L$} in the \emph{``basis''} $\{ \colvec{1\\0}, \colvec{0\\1} \}$.

Since every linear function from $\Re^2\rightarrow \Re^2$ can be given a matrix in this way, we see that every such linear function has a matrix in the basis $\{ \colvec{1\\0}, \colvec{0\\1} \}$. We will revisit this idea in depth later, and develop the notion of a basis further, and learn about how to make a matrix for an arbitrary linear transformation $\Re^n\rightarrow \Re^m$ in an arbitrary basis.
\end{remark}



\subsection{Invariant Directions}

Consider the linear transformation $L$ such that $$L\colvec{1\\0}=\colvec{-4\\-10}\,  \mbox{ and }\, L\colvec{0\\1}=\colvec{3\\7}\, ,$$ so that the matrix of $L$ is $\begin{pmatrix}
-4 & 3 \\
-10 & 7 \\
\end{pmatrix}$.  Recall that a vector is a direction and a magnitude; $L$ applied to $\colvec{1\\0}$ or $\colvec{0\\1}$ changes both the direction and the magnitude of the vectors given to it.

Notice that $L\colvec{3\\5}=\colvec{-4\cdot 3+3\cdot 5 \\ -10\cdot 3+7\cdot 5}=\colvec{3\\5}$.  Then $L$ fixes both the magnitude and direction of the vector $v_1=\colvec{3\\5}$.  {\it Try drawing a picture of this situation on some graph paper to help yourself visualize it better!}

Now, notice that any vector with the same direction as $v_1$ can be written as $cv_1$ for some constant $c$.  Then $L(cv_1)=cL(v_1)=cv_1$, so $L$ fixes every vector pointing in the same direction as $v_1$.

Also notice that $L\colvec{1\\2}=\colvec{-4\cdot 1+3\cdot 2 \\ -10\cdot 1+7\cdot 2}=\colvec{2\\4}=2\colvec{1\\2}$.  Then $L$ fixes the direction of the vector $v_2=\colvec{1\\2}$ but stretches $v_2$ by a factor of $2$.  Now notice that for any constant $c$, $L(cv_2)=cL(v_2)=2cv_2$.  Then $L$ stretches every vector pointing in the same direction as $v_2$ by a factor of $2$.

In short, given a linear transformation $L$ it is sometimes possible to find a vector $v\neq 0$ and constant $\lambda\neq 0$ such that 
\[
L(v)=\lambda v
\]

We call the direction of the vector $v$ an \emph{invariant direction}\index{Invariant direction}.  In fact, any vector pointing in the same direction also satisfies the equation:  $L(cv)=cL(v)=\lambda cv$.  The vector $v$ is called an \emph{eigenvector}\index{Eigenvector} of $L$, and $\lambda$ is an \emph{eigenvalue}\index{Eigenvalue}.  Since the direction is all we really care about here, then any other vector $cv$ (so long as $c\neq 0$) is an equally good choice of eigenvector.

Returning to our example of the linear transformation $L$ with matrix $\begin{pmatrix}
-4 & 3 \\
-10 & 7 \\
\end{pmatrix}$, we have seen that $L$ enjoys the property of having two invariant directions, represented by eigenvectors $v_1$ and $v_2$ with eigenvalues $1$ and $2$, respectively.  

It would be very convenient if I could write any vector $w$ as a linear combination of $v_1$ and $v_2$.  Suppose $w=rv_1+sv_2$ for some constants $r$ and~$s$.  Then:
\[
L(w)=L(rv_1+sv_2)=rL(v_1)+sL(v_2)=rv_1+2sv_2.
\]

Now $L$ just multiplies the number $r$ by $1$ and the number $s$ by $2$.  If we could write this as a matrix, it would look like:
\[
\begin{pmatrix}
1 & 0 \\
0 & 2
\end{pmatrix}\colvec{r\\s}
\]
which is much slicker than the usual scenario  $L\!\begin{pmatrix}
x\\
y
\end{pmatrix}\!=\!\begin{pmatrix}
\!a&b\! \\
\!c&d\!
\end{pmatrix} \! \!\begin{pmatrix}
x \\
y
\end{pmatrix}\!=\!
\begin{pmatrix}
\!ax+by\! \\
\!cx+dy\!
\end{pmatrix}$.
Here, $r$ and $s$ give the coordinates of $w$ in terms of the vectors $v_1$ and $v_2$.  In the previous example, we multiplied the vector by the matrix $L$ and came up with a complicated expression.  In these coordinates, we can see that~$L$ is a very simple \emph{diagonal matrix}, whose diagonal entries are exactly the \emph{eigenvalues} of $L$.

This process is called \emph{diagonalization}\index{Diagonalization!concept of}, and it can make complicated linear systems much easier to analyze.

Now that we've seen what eigenvalues and eigenvectors are, there are a number of questions that need to be answered.

\begin{itemize}
\item How do we find eigenvectors and their eigenvalues?
\item How many eigenvalues and (independent) eigenvectors does a given linear transformation have?
\item When can a linear transformation be diagonalized?
\end{itemize}
We'll start by trying to find the eigenvectors for a linear transformation.

\begin{example}
Let $L:\Re^2\rightarrow \Re^2$ such that $L(x,y)=(2x+2y, 16x+6y)$.  First, we can find the matrix of $L$:

\[
\colvec{x\\y}\stackrel{L}{\mapsto} \begin{pmatrix}
2 & 2 \\
16 & 6
\end{pmatrix}\colvec{x\\y}.
\]
We want to find an invariant direction $v=\colvec{x\\y}$ such that
\[
L(v)=\lambda v
\]
or, in matrix notation,
\begin{eqnarray*}
\begin{pmatrix}
2 & 2 \\
16 & 6
\end{pmatrix}\colvec{x\\y}&=&\lambda \colvec{x\\y} \\[2mm]
\Leftrightarrow \begin{pmatrix}
2 & 2 \\
16 & 6
\end{pmatrix}\colvec{x\\y}&=&\begin{pmatrix}
\lambda & 0 \\
0 & \lambda
\end{pmatrix} \colvec{x\\y} \\\\ 
\Leftrightarrow \begin{pmatrix}
2-\lambda & 2 \\
16 & 6-\lambda
\end{pmatrix}\colvec{x\\y}&=& \colvec{0\\0}.
\end{eqnarray*}
This is a homogeneous system, so it only has solutions when the matrix $\begin{pmatrix}
2-\lambda & 2 \\
16 & 6-\lambda
\end{pmatrix}$ is singular.  In other words, 
\begin{eqnarray*}
\det \begin{pmatrix}
2-\lambda & 2 \\
16 & 6-\lambda
\end{pmatrix}&=&0 \\[2mm]
\Leftrightarrow (2-\lambda)(6-\lambda)-32&=&0 \\[2mm]
\Leftrightarrow \lambda^2-8\lambda-20&=&0\\[2mm]
\Leftrightarrow (\lambda-10)(\lambda+2)&=&0
\end{eqnarray*}

For any square $n\times n$ matrix $M$, the polynomial in $\lambda$ given by $$P_M(\lambda)=\det (\lambda I-M)=(-1)^n \det (M-\lambda I)$$ is called the \emph{characteristic polynomial}\index{Characteristic polynomial} of $M$, and its roots are the eigenvalues of $M$.

In this case, we see that $L$ has two eigenvalues, $\lambda_1=10$ and $\lambda_2=-2$.  To find the eigenvectors, we need to deal with these two cases separately.
To do so, we solve the linear system $\begin{pmatrix}
2-\lambda & 2 \\
16 & 6-\lambda
\end{pmatrix}\colvec{x\\y}= \colvec{0\\0}$ with the particular eigenvalue $\lambda$ plugged in to the matrix.

\begin{itemize}
\item[$\underline{\lambda=10}$:]  We solve the linear system
\[
\begin{pmatrix}
-8 & 2 \\
16 & -4
\end{pmatrix}\colvec{x\\y}= \colvec{0\\0}.
\] 
Both equations say that $y=4x$, so any vector $\colvec{x\\4x}$ will do.  Since we only need the direction of the eigenvector, we can pick a value for $x$.  Setting $x=1$ is convenient, and gives the eigenvector $v_1=\colvec{1\\4}$.

\item[$\underline{\lambda=-2}$:]  We solve the linear system
\[
\begin{pmatrix}
4 & 2 \\
16 & 4
\end{pmatrix}\colvec{x\\y}= \colvec{0\\0}.
\] 
Here again both equations agree, because we chose $\lambda$ to make the system singular.  We see that $y=-2x$ works, so we can choose $v_2=\colvec{1\\-2}$.
\end{itemize}

In short, our process was the following:
\begin{itemize}
\item Find the characteristic polynomial of the matrix $M$ for $L$, given by\footnote{It is often easier (and equivalent if you only need the roots) to compute $\det(M-\lambda I)$.} $\det (\lambda I-M)$.
\item Find the roots of the characteristic polynomial; these are the eigenvalues of $L$.
\item For each eigenvalue $\lambda_i$, solve the linear system $(M-\lambda_i I)v=0$ to obtain an eigenvector $v$ associated to $\lambda_i$.
\end{itemize}
\end{example}



\section*{References}
\begin{itemize}
\item Hefferon, Chapter Three, Section III.1: Representing Linear Maps with Matrices
\item Hefferon, Chapter Five, Section II.3: Eigenvalues and Eigenvectors
\end{itemize}

Wikipedia:
\begin{itemize}
\item \href{http://en.wikipedia.org/wiki/Eigenvalue,_eigenvector_and_eigenspace}{Eigen*}
\item \href{http://en.wikipedia.org/wiki/Characteristic_polynomial}{Characteristic Polynomial}
\item \href{http://en.wikipedia.org/wiki/Linear_map}{Linear Transformations (and matrices thereof)}
\end{itemize}










\section*{Review Questions}

\begin{enumerate}
\item Consider $L: \Re^2\rightarrow \Re^2$ with $L(x,y)=(x\cos \theta +y\sin \theta, -x\sin \theta + y\cos \theta)$.
\begin{itemize}
\item[$i.$] Write the matrix of $L$ on the basis $\colvec{1\\0}, \colvec{0\\1}$.

\item[$ii.$] When $\theta\neq 0$, explain how $L$ acts on the plane.  Draw a picture.

\item[$iii.$] Do you expect $L$ to have invariant directions?

\item[$iv.$] Try to find eigenvalues for $L$ by solving the equation 
\[
L(v)=\lambda v.
\]

\item[$v.$] Does $L$ have real eigenvalues?  If not, are there complex eigenvalues for $L$, assuming that $i=\sqrt{-1}$ exists?
\end{itemize}

\item Let $M=\begin{pmatrix}
2 & 1 \\
0 & 2
\end{pmatrix}$.  Find all eigenvalues of $M$.  Does $M$ have two independent eigenvectors?  Can $M$ be diagonalized?

\item Let $L$ be the linear transformation $L:\Re^3\rightarrow \Re^3$ given by $L(x,y,z)=(x+y, y+z, x+z)$.  Let $e_i$ be the vector with a one in the $i$th position and zeros in all other positions.
\begin{itemize}
\item[$i.$] Find $Le_i$ for each $i$.
\item[$ii.$] Given a matrix $M=\begin{pmatrix}
m^1_1 & m^1_2 & m^1_3\\
m^2_1 & m^2_2 & m^2_3\\
m^3_1 & m^3_2 & m^3_3\\
\end{pmatrix}$, what can you say about $Me_i$ for each $i$?
\item[$iii.$] Find a $3\times 3$ matrix $M$ representing $L$.  Choose three non-trivial vectors pointing in different directions and show that $Mv=Lv$ for each of your choices.
\end{itemize}

\end{enumerate}


\newpage