Important Topics for the Final Exam

This set of notes is an extension of the notes provided for the first midterm exam, and is a guideline for topics to be studied for the in-class portion of the final exam. It should be noted that this list is not all inclusive, and any topic covered throughout the semester could appear on the final exam.

Section 1.

The following definitions should be memorized:

Definition 1   Let $\v _1,\dots,\v _k$ be vectors in a vector space $V$. Then the span of the set $\{\v _1,\dots,\v _k\}$ is defined to be the set of all linear combinations of the vectors $\v _1,\dots,\v _k$. That is,

$$span\{\v _1,\dots,\v _k\}= \left\{c_1\v _1+c_2+\v _2+\cdots+c_k\v _k:c_i\in\mathbb{R}\right\}$$

Definition 2   An indexed set of vectors $\{\v _1,\dots,\v _p\}$ in a vector space $V$ is sadi to be linearly independent if the vector equation

$$x_1\v _1+x_2\v _2+\cdots+x_p\v _p=\bar{0}$$

has only the trivial solution. The set $\{\v _1,\dots,\v _p\}$ is said to be linearly dependent if there exist wieghts $c_1,\dots,c_p$, not all zero, such that

$$c_1\v _1+c_2\v _2+\cdots+c_p\v _p=\bar{0}$$

Definition 3   A transformation $T$ is linear provided:

(i)

$T(\u +\v )=T(\u )+T(\v )$ for all $\u ,\v$ in the domain of $T$;

(ii)

$T(c\u )=cT(\u )$ for all scalars $c$ and $\u$ in the domain of $T$.

Definition 4   A mapping $T:\mathbb{R}^n\mapsto\mathbb{R}^m$ is said to be one-to-one if each $\b\in\mathbb{R}^m$ is the image of at most one vector $\bar{x}\in\mathbb{R}^n$.

Definition 5   A mapping $T:\mathbb{R}^n\mapsto\mathbb{R}^m$ is said to be onto if each $\b\in\mathbb{R}^m$ is the image of at least one vector $\bar{x}\in\mathbb{R}^n$.

Remark: The concept of onto relates to the existence of solutions, and the notion of one-to-one relates to the uniqueness of solutions.

Definition 6   If $A$ is an $m\times n$ matrix, and $B$ is an $n\times p$ matrix with columns $\b _1,\dots,\b _p$, then the product $AB$ is the $m\times p$ matrix whose columns are $A\b _1,\dots, A\b _p$. That is,

$$AB=A[\,\b _1\;\b _2\,\cdots\,\b _p\,]=[\,A\b _1\;\,A\b _2\;\,\cdots\;\,A\b _p\,]$$

Remark: Suppose that $A=[\bar{a}_1\;\bar{a}_2\;\cdots\;\bar{a}_n]$ is an $m\times n$ matrix. Then given this definition of matrix multiplication, when we talk of the span of the columns of $A$, we can say

$$span\{\bar{a}_1,\bar{a}_2,\dots,\bar{a}_n\} = \{x_1\bar{a}_1+x_2\bar{a}_2+\cdots+x_n\bar{a}_n:x_i\in\mathbb{R}\}$$

$$= \{A\bar{x}:\bar{x}\in\mathbb{R}^n\}$$

So a vector $\b$ is in the span of the columns of $A$ if and only if $A\bar{x}=\b$ is consistent.

Definition 7   A mapping is invertible if and only if it is both one-to-one and onto.

Definition 8   Suppose $T:V\rightarrow W$ is a linear transformation. Then the null space of $T$, denoted $null\;T$, is defined to be

$$null\;T=\{\v\in V:T(\v )=\bar{0}\}.$$

The range of $T$, denoted $range\;T$, is defined to be

$$range\;T=\{T(\v ):v\in W\}.$$

That is, the range of $T$ is the set of all images of $T$.

Definition 9   Let $U$ be a vector space. A basis for $U$ is a linearly independent spanning set of $U$.

Definition 10   Let $V$ be a vector space. The dimension of $V$ is defined to be the number of vectors in a basis for $V$.

Definition 11   Consider a $2\times 2$ matrix $A=\begin{pmatrix} a&b\\ c&d\\ \end{pmatrix}$. We define the determinant of $A$ as

$$det(A)=ad-bc$$

Definition 12   Let $A=(a_{i,j})$ be an $n\times n$ matrix, and let $A_{i,j}$ be the $(n-1)\times(n-1)$ matrix formed by deleting the $i$th row and $j$th column from $A$. Then we (recursively) define the determinant of $A$ as

$$det(A) = \displaystyle \sum^{n}_{j=1}a_{1,j}\cdot det(A_{1,j})$$

$$= a_{1,1}\cdot det(A_{1,1})+a_{1,2}\cdot det(A_{1,2})+\cdots+a_{1,n}\cdot det(A_{1,n})$$

Remark: The determinant of $A$ can be expanded about any row or column of $A$.

Definition 13   Suppose $A$ is an $n\times n$ matrix. Then a non-zero vector $\v$ is an eigenvector of $A$ provided $A\v =\lambda\v$ for some scalar $\lambda$. The scalar $\lambda$ is called an eigenvalue of $A$.

Remark: Even though we only allow non-zero vectors to be eigenvectors, the scalar $\lambda=0$ is allowed to be an eigenvalue of $A$.

Definition 14   Let $A$ be an $n\times n$ matrix. The characteristic polynomial of $A$ is defined to be $det(A-\lambda I)$.

Definition 15   Let $V$ be a vector space (over $\mathbb{R}$). An inner product on $V$ is a function $\langle,\rangle:V\times V\rightarrow\mathbb{R}$ satisfying the following properties:

(i)

$\langle\u ,\v\rangle=\langle\v ,\u\rangle$ for all $\u ,\v\in V$.

(ii)

$\langle c\u ,\v\rangle=c\langle\u ,\v\rangle$ for all $u\in V$ and all $c\in\mathbb{R}$.

(iii)

$\langle\u +\v ,\bar{w}\rangle=\langle\u ,\bar{w}\rangle+\langle\v ,\bar{w}\rangle$ for all $\u ,\v ,\bar{w}\in V$.

(iv)

$\langle \v ,\v\rangle\ge0$ for all $\v\in V$, with equality if and only if $\v =\bar{0}$.

Definition 16   Let $V$ be a vector space with inner product $\langle,\rangle:V\times V\rightarrow\mathbb{R}$. A set of non-zero vectors $\{\u _1,\dots,\u _p\}$ is an orthogonal set (with respect to the given inner product) if

$$\langle\u _i,\u _j\rangle = \left\{ \begin{array}{r@{\quad:\quad}l} 0&i\not=j\\ \not=0&i=j\\ \end{array} \right.$$

The set is called an orthonormal set if

$$\langle\u _i,\u _j\rangle = \left\{ \begin{array}{r@{\quad:\quad}l} 0&i\not=j\\ 1&i=j\\ \end{array} \right.$$

Section 2.

You must memorize and understand the following theorems:

Theorem 1   Each matrix is row equivalent to one and only one reduced echelon matrix.

Theorem 2   A linear system is consistent if and only if the rightmost column of the augmented matrix is not a pivot column. That is, if and only if an echelon form of the augmented matrix has no row of the form

$$[\,0\;0\,\cdots\,0\;b],\;\;b\not=0$$

If a linear system is consistent, the there is either a unique solution or infinitely many solutions.

Remark: A system of linear equations has either zero, one, or infinitely many solutions.

Theorem 3   The homogeneous equation $A\bar{x}=\bar{0}$ has a non-trivial solution if and only if the equation has a free variable - that is, if and only if $A$ has a non-pivot column.

Theorem 4   The columns of a matrix $A$ are linearly independent if and only if the equation $A\bar{x}=\bar{0}$ has only the trivial solution - that is, if and only if $A$ has a pivot in every column.

Theorem 5   Let $T:\mathbb{R}^n\mapsto\mathbb{R}^m$ be a linear transformation and let $A$ be the standard matrix for $T$. Then

(i)

$T$ is one-to-one if and only if $A$ has a pivot in every column.

(ii)

$T$ is onto if and only if $A$ has a pivot in every row.

Theorem 6   Let $T:\mathbb{R}^n\mapsto\mathbb{R}^n$ be a linear transformation and let $A$ be the standard matrix for $T$. Then $T$ is invertible (and also $A$ is invertible) if and only if one of the following conditions is true:

1. $A$ is row equivalent to the identity matrix.

2. The equation $A\bar{x}=\bar{0}$ has only the trivial solution.

3. The mapping $\bar{x}\mapsto A\bar{x}$ is one-to-one.

4. $dim\;null\;T=0$.

5. The equation $A\bar{x}=\b$ has at least one solution for all $\b\in\mathbb{R}^n$.

6. The mapping $\bar{x}\mapsto A\bar{x}$ is onto.

7. $dim\;range\;T$ equals the dimension of the codomain of $T$.

8. There is a matrix $B$ such that $AB=I=BA$.

9. $det\;A\not=0$.
   

Theorem 7   A square matrix $A$ is NOT invertible if and only if $\lambda=0$ is an eiegenvalue of $A$.

Theorem 8   Let $T:V\rightarrow W$ be a linear transformation. Then

$$dim\;null\;T+dim\;range\;T=dim\;V$$

Theorem 9   Let $\{\u _1,\dots,\u _p\}$ be vectors in $\mathbb{R}^n$, and form the matrix $U=[\,u_1\,\cdots\,\u _p\,]$. Then the set is an orthogonal set (with respect to the dot product) if and only if $U^TU$ is a diagonal matrix. The set is an orthonormal set if and only if $U^TU=I$.

Theorem 10   Let $\{\u _1,\dots,\u _p\}$ be an orthogonal set of vectors. Then the set is also a linearly independent set of vectors.

Theorem 11   Let $\{\u _1,\dots,\u _p\}$ be an orthogonal basis for a subspace $U$ of the vector space $V$.

(i)

For all $\u\in U$,

$$\u = \left(\displaystyle \frac{\langle\u ,\u _1\rangle}{\langle\u... ...playstyle \frac{\langle\u ,\u _p\rangle}{\langle\u _p,\u _p\rangle}\right)\u _p$$

Remark: If the set is an orthonormal basis, then

$$\u =(\langle\u ,\u _1\rangle)\u _1+(\langle\u ,\u _2\rangle)\u _2+\cdots +(\langle\u ,\u _p\rangle)\u _p$$

(ii)

For all $\v\in V$, the orthogonal projection of $\v$ onto $U$ is found as

$$Proj_{_U}\v = \left(\displaystyle \frac{\langle\v ,\u _1\rangle}{... ...playstyle \frac{\langle\v ,\u _p\rangle}{\langle\u _p,\u _p\rangle}\right)\u _p$$

Remark: If the set is an orthonormal basis, then

$$Proj_{_U}\v =(\langle\v ,\u _1\rangle)\u _1+(\langle\v ,\u _2\rangle)\u _2+\cdots +(\langle\v ,\u _p\rangle)\u _p.$$

Moreover, if $V=\mathbb{R}^n$, and if we form the matrix $U_{_M}$, we get

$$Proj_{_U}\v =U_{_M}U^T_{_M}v$$

(iii)

If $A$ is a matrix and the set is an orthonormal basis for $Col\;A$, then we can find a $QR$-factorization of $A$, where $Q=[\;\u _1\,\cdots\,\u _p\,]$, and $R=Q^TA$ is an upper triangular matrix.

Theorem 12 (Gram-Schmidt)   Let $V$ be a vector space with an inner product, and suppose $\{\u _1,\dots,\u _p\}$ is a basis for a subspace $U$. Form the vectors

$$\v _1 = \u _1$$

$$\v _2 = \u _2- \left(\displaystyle \frac{\langle\u _2,\v _1\rangle}{\langle\v _1,\v _1\rangle}\right)\v _1$$

$$\v _3 = \u _3- \left(\displaystyle \frac{\langle\u _3,\v _1\rangle}{\lang... ...aystyle \frac{\langle\u _3,\v _2\rangle}{\langle\v _2,\v _2\rangle}\right)\v _2$$

$$\vdots$$

$$\v _p = \u _p- \left(\displaystyle \frac{\langle\u _p,\v _1\rangle}{\lang... ...gle\u _p,\v _{p-1}\rangle} {\langle\v _{p-1},\v _{p-1}\rangle}\right)\v _{p-1}.$$

Then $\{\v _1,\dots,\v _p\}$ is an orthogonal basis for $U$.

Remark: This gaurantees that every vector space with an inner product has an orthogonal (and so also an orthonormal) basis with respect to the given inner product.

Theorem 13   Let $V$ be an inner product space and $W$ a subspace of $V$. Then, for all $\v\in V$, the orthogonal projection of $\v$ onto $W$ is the vector in $W$ that is ``closest'' (with respect to the given inner product) to $\v$. That is,

$$\vert\vert\v -Proj_{_W}\v \vert\vert\le\vert\vert\v -\bar{w}'\vert\vert$$

for all $\bar{w}'\in W$

Remark: If $\v\in W$, then $Proj_{_W}\v =\v$.

Theorem 14   Suppose that $A\bar{x}=\b$ has no solution. Then a least squares solution is a solution $\hat{x}$ to $(A^TA)\hat{x}=(A^T\b )$.

Remark: To find a least squares solution to $A\bar{x}=\b$ we do NOT need to find the projection of $\b$ onto $Col\;A$.

Section 3.

You should be able to perform each of the following:

(1)

Row reduce a matrix to REF and RREF.

(2)

Find the parametric form of solutions to a linear system.

(3)

Find the parametric vector form of solutions to a matrix-vector equation $A\bar{x}=\b$.

(4)

Determine if a matrix is invertible.

(5)

Find the inverse of an invertible matrix.

(6)

Determine the domain and codomain of a matrix transformation.

(7)

Determine if a set of vetors in $\mathbb{R}^n$ is linearly independent.

(8)

Decide if a transformation is a linear transformation.

(9)

Find the standard matrix of a linear transformation $T$.

(10)

Find the matrix of a linear transformation $T$ with respect to any basis.

(11)

Find a basis for the null space of a linear transformation.

(12)

Find a basis for the range of a linear transformation.

(13)

Determine the rank of a matrix.

(14)

Find the characteristic polynomial of a matrix $A$, and hence find all the eigenvalues of a matrix $A$.

(15)

Find a basis for an eigenspace of a matrix $A$.

(16)

Give a $PDP^{-1}$-factorization of a matrix $A$, and decide whether this is possible for a specific matrix.

(17)

Determine whether a set of vectors is an orthogonal/orthonormal set with respect to a given inner product.

(18)

Use the Gram-Schmidt algorithm to find an orthogonal/orthonormal basis for a subspace.

(19)

Find a least squares solution to an inconsistent system $A\bar{x}=\b$.

(20)

Find an orthogonal projection of a vector onto a subspace $W$.

(21)

Find the $QR$-factorization of a matrix $A$.

(22)

Prove elementary facts about linear algebra.

Section 4.

Some Examples.

Example 1.

Let $T:\mathbb{R}^2\mapsto\mathbb{R}^2$ be the map that projects points onto the $x_1$-axis.

Although not necessary, a formula for $T$ would be

$$T:\begin{pmatrix} x\\ y\\ \end{pmatrix}\mapsto\begin{pmatrix} x\\ 0\\ \end{pmatrix}$$

We first prove that $T$ is a linear transformation. Choose two arbitrary vectors

$$\u =\begin{pmatrix} a\\ b\\ \end{pmatrix}\;\;\;\u =\begin{pmatrix} c\\ d\\ \end{pmatrix}$$

Then,

$$T(\u +\v ) = T\left[\begin{pmatrix} a\\ b\\ \end{pmatrix}+\begin{pmatrix} c\\ d\\ \end{pmatrix}\right]$$

$$= T\begin{pmatrix} a+c\\ b+d\\ \end{pmatrix}$$

$$= \begin{pmatrix} a+c\\ 0\\ \end{pmatrix}$$

$$T(\u )+T(\v ) = \begin{pmatrix} a\\ 0\\ \end{pmatrix}\begin{pmatrix} c\\ 0\\ \end{pmatrix}$$

$$= \begin{pmatrix} a+c\\ 0\\ \end{pmatrix}$$

So $T(\u +\v )=T(\u )+T(\v )$. Next we choose an arbitrary scalar $k$ and compute the following.

$$T(k\u ) = T\begin{pmatrix} ka\\ kb\\ \end{pmatrix}$$

$$= \begin{pmatrix} ka\\ 0\\ \end{pmatrix}$$

$$kT(\u ) = k\begin{pmatrix} a\\ 0\\ \end{pmatrix}$$

$$= \begin{pmatrix} ka\\ 0\\ \end{pmatrix}$$

So $T(k\u )=kT(\u )$. We have shown that $T$ is a linear transformation.

Now we find the standard matrix for $T$. Recall that $A=[\;T(\bar{e}_1)\,\;T(\bar{e}_2)\;]$.

$$T(\bar{e}_1)=T\begin{pmatrix} 1\\ 0\\ \end{pmatrix}=\begin{pma... ...egin{pmatrix} 0\\ 1\\ \end{pmatrix}=\begin{pmatrix} 0\\ 0\\ \end{pmatrix}$$

So the standard matrix is

$$A=\begin{pmatrix} 1&0\\ 0&0\\ \end{pmatrix}$$

See that $A$ is already in RREF. So since $A$ does not have a pivot in every column the map $T$ is NOT one-to-one. Since $A$ does not have a pivot in every row the map $T$ is NOT onto. Therefore, $T$ is NOT invertible.

We take a geometric view of this. To see that $T$ is not one-to-one notice that the point $p$ on the $x_1$-axis is the image of infinitely many points in $\mathbb{R}^2$. There are precisely all the points on the lines $l_1$ and $l_2$ in the following diagram..

To see that $T$ is not onto, notice that any point of $\mathbb{R}^2$ off of the $x_1$-axis is not the image of any point in $\mathbb{R}^2$. See this for the point $\b$ in the following diagram.

This example ties together many important ideas.

Example 2.

Find a $QR$-factorization of

$$A=\begin{pmatrix} 1&0&0\\ 1&1&0\\ 1&1&1\\ 1&1&1\\ \end{pmatrix}.$$

(i)

Find an orthogonal basis for $Col A=span\{\bar{a}_1,\bar{a}_2,\bar{a}_3\}$.

We use the Gram-Schmidt algorithm and form the following vectors:

$$\v _1 = \bar{a}_1=\begin{pmatrix} 1\\ 1\\ 1\\ 1\\ \end{pmatrix}$$

$$\v _2 = \bar{a}_2-\left(\displaystyle \frac{\bar{a}_2\cdot\v _1}{\v _1\cdot\v _1}\right)\v _1$$

$$= \begin{pmatrix} 0\\ 1\\ 1\\ 1\\ \end{pmatrix}-\left(\displaystyl... ... 1\\ 1\\ \end{pmatrix}=\begin{pmatrix} -3/4\\ 1/4\\ 1/4\\ 1/4\\ \end{pmatrix}$$

Since scaling vectors does not affect orthogonality, (for computational ``simplicity'') we replace $\v _2$ with the vector

$$\v _2=\begin{pmatrix} -3\\ 1\\ 1\\ 1\\ \end{pmatrix}.$$

Then

$$\v _3 = \bar{a}_3-\left(\displaystyle \frac{\bar{a}_3\cdot\v _1}{\v _1\cd... ..._1- \left(\displaystyle \frac{\bar{a}_3\cdot\v _2}{\v _2\cdot\v _2}\right)\v _2$$

$$= \begin{pmatrix} 0\\ 0\\ 1\\ 1\\ \end{pmatrix}-\left(\displaystyl... ...displaystyle \frac{2}{12}\right)\begin{pmatrix} -3\\ 1\\ 1\\ 1\\ \end{pmatrix}$$

$$= \begin{pmatrix} 0\\ -2/3\\ 1/3\\ 1/3\\ \end{pmatrix}$$

So the set

$$\left\{\begin{pmatrix} 1\\ 1\\ 1\\ 1\\ \end{pmatrix},\begin{... ...\end{pmatrix},\begin{pmatrix} 0\\ -2/3\\ 1/3\\ 1/3\\ \end{pmatrix}\right\}$$

is an orthogonal basis for $Col\;A$.

(ii)

Find an orthonormal basis for $Col\;A$.

We simply scale each vector from part (i) so that each has norm equal to one. Since $\vert\vert\v _1\vert\vert=2$, $\vert\vert\v _2\vert\vert=\sqrt{12}$, and $\vert\vert\v _3\vert\vert=\sqrt{2/3}$ we get

$$\left\{\begin{pmatrix} 1/2\\ 1/2\\ 1/2\\ 1/2\\ \end{pmatrix}... ...pmatrix} 0\\ -2/\sqrt{6}\\ 1/\sqrt{6}\\ 1/\sqrt{6}\\ \end{pmatrix}\right\}$$

is an orthonormal basis for $Col\;A$.

(iii)

Form the matrix $Q$.

The columns of $Q$ are the orthonormal basis for $Col\;A$. So

$$Q=\begin{pmatrix} 1/2&-3/\sqrt{12}&0\\ 1/2&1/\sqrt{12}&-2/\sqrt... ...\ 1/2&1/\sqrt{12}&1/\sqrt{6}\\ 1/2&1/\sqrt{12}&1/\sqrt{6}\\ \end{pmatrix}$$

(iv)

Find the matrix $R$.

We find $R$ as $R=Q^TA$.

$$R=\begin{pmatrix} 1/2&1/2&1/2&1/2\\ -3/\sqrt{12}&1/\sqrt{12}&1/... ...rix} 2&3/2&1\\ 0&3/\sqrt{12}&2/\sqrt{12}\\ 0&0&2/\sqrt{6}\\ \end{pmatrix}$$

We have found the $QR$-factorization

$$A=\begin{pmatrix} 1&0&0\\ 1&1&0\\ 1&1&1\\ 1&1&1\\ \end{pm... ...rix} 2&3/2&1\\ 0&3/\sqrt{12}&2/\sqrt{12}\\ 0&0&2/\sqrt{6}\\ \end{pmatrix}$$

Example 3.

Let $\u _1=\begin{pmatrix} 1\\ 1\\ 0\\ -1\\ \end{pmatrix}$, $\u _2=\begin{pmatrix} 1\\ 0\\ 1\\ 1\\ \end{pmatrix}$, $\u _3=\begin{pmatrix} 0\\ -1\\ 1\\ -1\\ \end{pmatrix}$, $\b =\begin{pmatrix} 2\\ 5\\ 6\\ 6\\ \end{pmatrix}$. Find the orthogonal projection of $\b$ onto the subspace $W=span\{\u _1,\u _2,\u _3\}$.

(i)

We already have an orthogonal basis for $W$.

$$\left\{\u _1=\begin{pmatrix} 1\\ 1\\ 0\\ -1\\ \end{pmatrix},... ...end{pmatrix},\u _3=\begin{pmatrix} 0\\ -1\\ 1\\ -1\\ \end{pmatrix}\right\}$$

It follows that

$$\left\{\v _1=\begin{pmatrix} 1/\sqrt{3}\\ 1/\sqrt{3}\\ 0\\ -1/... ...matrix} 0\\ -1/\sqrt{3}\\ 1/\sqrt{3}\\ -1/\sqrt{3}\\ \end{pmatrix}\right\}$$

is an orthonormal basis for $W$.

(ii)

We find the projection of $\b$ onto $W$ in two different ways.

(a)

Using the orthogonal basis for $W$ and Theorem from section 2 we get

$$Proj_{_W}\b = \left(\displaystyle \frac{\langle\b ,\u _1\rangle} {\langle\u _1,... ...playstyle \frac{\langle\b ,\u _3\rangle}{\langle\u _3,\u _3\rangle}\right)\u _3$$

$$= \left(\displaystyle \frac{1}{3}\right)\begin{pmatrix} 1\\ 1\\ 0\\... ...isplaystyle \frac{-5}{3}\right)\begin{pmatrix} 0\\ -1\\ 1\\ -1\\ \end{pmatrix}$$

$$= \begin{pmatrix} 1/3\\ 1/3\\ 0\\ -1/3\\ \end{pmatrix}+\begin{pmat... ...3\\ 14/3\\ \end{pmatrix}+\begin{pmatrix} 0\\ 5/3\\ -5/3\\ 5/3\\ \end{pmatrix}$$

$$= \begin{pmatrix} 5\\ 2\\ 3\\ 6\\ \end{pmatrix}$$

(b)

Using the orthonormal basis and Theorem again we get

$$Proj_{_W}\b = UU^T\b$$

$$= \begin{pmatrix} 1/\sqrt{3}&1/\sqrt{3}&0\\ 1/\sqrt{3}&0&-1/\sqrt{... ...t{3}&-1/\sqrt{3}\\ \end{pmatrix}\begin{pmatrix} 2\\ 5\\ 6\\ 6\\ \end{pmatrix}$$

$$= \begin{pmatrix} 5\\ 2\\ 3\\ 6\\ \end{pmatrix}$$

Example 4.

Find a least squares solution to $A\bar{x}=\b$ where

$$A=\begin{pmatrix} 1&1&0\\ 1&0&-1\\ 0&1&1\\ -1&1&-1\\ \end{pmatrix}\;\;\;$$and$$\;\;\;\b =\begin{pmatrix} 2\\ 5\\ 6\\ 6\\ \end{pmatrix}$$

We solve this in two different ways.

(a)

We solve $A\bar{x}=\b ^*$, where $\b ^*$ is the orthogonal projection of $\b$ onto $Col\;A$.

In the previous problem we found that

$$\b ^*=Proj_{Col\;A}\b =\begin{pmatrix} 5\\ 2\\ 3\\ 6\\ \end{pmatrix}$$

So we solve

$$\begin{pmatrix} 1&1&0\\ 1&0&-1\\ 0&1&1\\ -1&1&-1\\ \end{pmatrix}\bar{x}=\begin{pmatrix} 5\\ 2\\ 3\\ 6\\ \end{pmatrix}.$$

Row reduction gives us

$$\begin{pmatrix} 1&1&0&5\\ 1&0&-1&2\\ 0&1&1&3\\ -1&1&-1&6\\ ... ...pmatrix} 1&0&0&1/3\\ 0&1&0&14/3\\ 0&0&1&-5/3\\ 0&0&0&0\\ \end{pmatrix},$$

and the least squares solution is

$$\bar{x}=\begin{pmatrix} 1/3\\ 14/3\\ -5/3\\ \end{pmatrix}$$

(b)

Using Theorem we solve the system $(A^TA)\bar{x}=(A^T\b )$. We compute

$$A^TA=\begin{pmatrix} 1&1&0&-1\\ 1&0&1&1\\ 0&-1&1&-1\\ \end{... ...1\\ \end{pmatrix}=\begin{pmatrix} 3&0&0\\ 0&3&0\\ 0&0&3\\ \end{pmatrix}$$

and

$$A^T\b =\begin{pmatrix} 1&1&0&-1\\ 1&0&1&1\\ 0&-1&1&-1\\ \en... ... 5\\ 6\\ 6\\ \end{pmatrix}=\begin{pmatrix} 1\\ 14\\ -5\\ \end{pmatrix}$$

Then we solve

$$\begin{pmatrix} 3&0&0\\ 0&3&0\\ 0&0&3\\ \end{pmatrix}\bar{x}=\begin{pmatrix} 1\\ 14\\ -5\\ \end{pmatrix}.$$

This gives the least squares solution

$$\bar{x}=\begin{pmatrix} 1/3\\ 14/3\\ -5/3\\ \end{pmatrix}$$

Example 6.

If possible, find a $PDP^{-1}$-factorization of the matrix

$$A=\begin{pmatrix} -1&0&1\\ -3&4&1\\ 0&0&2\\ \end{pmatrix}$$

(i)

We find the eigenvalues of $A$ as the roots of the characteristic polynomial.

$$det(A-\lambda I) = det\begin{pmatrix} -1-\lambda&0&1\\ -3&4-\lambda&1\\ 0&0&2-\lambda\\ \end{pmatrix}$$

$$= -\lambda^3+5\lambda^2-2\lambda-8$$

$$= -(\lambda-4)(\lambda-2)(\lambda+1)$$

So the eigenvalues of $A$ are $\lambda=-1,2,4$. Since there are 3 distinct eigenvalues, $A$ must be diagonalizeable.

(ii)

We find a basis for each eigenspace, and recall that the dimension of each eigenspace is less than or equal to the algebraic multiplicity of that eigenvalue. So each eigenspace will have dimension one.

(a)

$\lambda=-1$: We find a basis for $null(A-\lambda I)=null(A+I)$.

$$A+I=\begin{pmatrix} 0&0&1\\ -3&5&1\\ 0&0&3\\ \end{pmatrix}\sim\begin{pmatrix} 1&-5/3&0\\ 0&0&1\\ 0&0&0\\ \end{pmatrix}$$

It follows that a basis for this eigenspace is

$$\left\{\begin{pmatrix} 5\\ 3\\ 0\\ \end{pmatrix}\right\}$$

(b)

$\lambda=2$: We find a basis for $null(A-\lambda I)=null(A-2I)$.

$$A-2I=\begin{pmatrix} -3&0&1\\ -3&2&1\\ 0&0&0\\ \end{pmatrix}\sim\begin{pmatrix} 1&0&-1/3\\ 0&1&0\\ 0&0&0\\ \end{pmatrix}$$

It follows that a basis for this eigenspace is

$$\left\{\begin{pmatrix} 1\\ 0\\ 3\\ \end{pmatrix}\right\}$$

(c)

$\lambda=4$: We find a basis for $null(A-\lambda I)=null(A-4I)$.

$$A-4I=\begin{pmatrix} -5&0&1\\ -3&0&1\\ 0&0&-2\\ \end{pmatrix}\sim\begin{pmatrix} 1&0&0\\ 0&0&1\\ 0&0&0\\ \end{pmatrix}$$

It follows that a basis for this eigenspace is

$$\left\{\begin{pmatrix} 0\\ 1\\ 0\\ \end{pmatrix}\right\}$$

(iii)

We form the matrix $P$ and the matrix $D$.

$$P=\begin{pmatrix} 5&1&0\\ 3&0&1\\ 0&3&0\\ \end{pmatrix}\;\;\;\;\;\;D=\begin{pmatrix} -1&0&0\\ 0&2&0\\ 0&0&4\\ \end{pmatrix}$$

(iv)

We find $P^{-1}$.

$$\begin{pmatrix} 5&1&0&1&0&0\\ 3&0&1&0&1&0\\ 0&3&0&0&0&1\\ \... ...rix} 1&0&0&1/5&0&-1/15\\ 0&1&0&0&0&1/3\\ 0&0&1&-3/5&1&1/5\\ \end{pmatrix}$$

So we get

$$P^{-1}=\begin{pmatrix} 1/5&0&-1/15\\ 0&0&1/3\\ -3/5&1&1/5\\ \end{pmatrix}$$

We have found the $PDP^{-1}$-factorization:

$$A=\begin{pmatrix} -1&0&1\\ -3&4&1\\ 0&0&2\\ \end{pmatrix}=\... ...matrix}\begin{pmatrix} 1/5&0&-1/15\\ 0&0&1/3\\ -3/5&1&1/5\\ \end{pmatrix}$$