Math 5660/CS 5606 - Numerical Analysis I, Fall 2001
Math 5661/CS 5616 - Numerical Analysis II, Spring 2002
Department of Mathematics
University of Colorado at Denver
This web page http://www-math.cudenver.edu/~jmandel/classes/5660f01
serves as the class syllabus and source of further information. Check it
regularly during the semester for further information.
Important notes will be put here during the semester - please check
frequently
PLEASE MAKE SURE THAT YOUR FILES ON math/linus ARE ACCESSIBLE SO THAT YOUR HOMEWORK CAN BE GRADED.
Class location and time: NC 2001, Monday and Wednesday 4-5:15
Office hours: CU Bldg 640, Monday 3:00-3:45 Wednesday 2:00-3:45
Prerequisites:
-
Math 3191 Linear Algebra (for both semesters): span, linear independence,
basis, norm, linear maps and matrices, operations with matrices, determinant,
systems of linear algebraic equations (abstract properties only, echelon
form not needed).
-
Math 1401 Calculus, especially the definition of limit, derivative, and integral.
Note that Math 1401 is prerequisite for Math 2411, which is prerequisite for Math 3191.
- Programming experience.
-
Math 3200 Differential Equations (Spring semester only): first order ordinary
differential equations and systems (basic properties only, specific solution
methods not needed), linear ordinary differential equations and systems
(general form of solution, homogeneous solution in the case of constant
coefficients)
Textbook: Rainer
Kress, Numerical Analysis, Springer 1998
Class format: The class will follow quite closely the book, except
for some chapters and sections that will be omitted. Students will be expected
to understand and to be able to reproduce the proofs with the help of the
text, solve (with a hint) problems that require the use of a part
of a proof from the text, write simple Matlab programs demonstrating the
algorithms, and produce variants of the algorithms.
Grading: There will be two in-class written tests
on October 17
and December 10 in Fall and March 11 and May 13 in Spring.
There will be a homework
every week or two, both theoretical an on the computer. All work will be
graded on the scale 4=perfect, 3=a minor problem, 2=good progress, 1=some
progress, 0=no progress or did not understand the assignment. Homeworks
turned in late count 50%. No credit for homeworks turned in so late that
the others are graded or even given back already. No guarantees - homeworks that
do not go with the pack might get misplaced. There is no extra credit. The
grade will be computed by scaling the total number of points from test
1, test 2, and all homeworks between 0 and 100 (100 is the best in the
class) and averaging the best two of the three. This formula allows for
a good grade for students who may miss the midterm, so no make-ups for the
midterm. Letter gradesA >=
90, A- >= 85, B+ >= 80, B >= 75, B- >=70, C+ >= 65, C>= 60,
C- >= 55, D+ >= 50, D >= 45, D- >= 40.
Computer assignments will be done in Matlab. Matlab is available
on MSWindows machines in the Zenas Hartvigson Lab SI 124 (subject to availability)
and on math.cudenver.edu, where everyone will get an account. See
instructions how to log in the first time and change your password. After
that, Linus can
be accessed from graphics terminals on the 6th floor of the CU-Denver building,
or in text mode (sufficient for most work) from any computer on the Internet
by SSH or as http://math.cudenver.edu.
Graphics over the Internet requires that SSH and Xwindows are installed
on your computer (this is standard in Linux) and preferably a connection
faster than phone line. The student edition of Matlab is also available
from the bookstore, but not necessary for this class. All homework files
are to be left in your math account. Files created on MSWindows
machines need to be transfered to math by sftp (secure file transfer,
part of SSH), and tested on math.You need to verify that your computer
homework runs correctly on math. See my Matlab
links for Matlab documentation.
Files for my Matlab demonstrations in the class can be found at
http://www-math.cudenver.edu/~jmandel/classes/5660f01/matlab
or you can copy them on math from /home/faculty/jmandel/public_html/classes/5660f01/matlab .
Topics covered:
Math 5660 Fall 2001
-
Introduction to Matlab: matrix operations, basic script programming and
graphing
-
Chapter 2 - linear systems: Gaussian elimination, LU decomposition, QR
decomposition
-
Chapter 3 - basic functional analysis: normed linear spaces, scalar product
and orthogonalization, bounded linear operators, matrix norms, competeness,
the Banach fixed point theorem, best approximation and projection
-
Chapter 4 - iterative methods for linear systems: Jacobi and Gauss-Seidel,
relaxation method
-
Chapter 6 - iterative methods for nonlinear systems: successive approximations,
Newton's method, zeros of polynomials and Horner's scheme
-
Chapter 8 - interpolation: Lagrange and Hermite polynomial interpolation,
splines
Math 5661 Spring 2002
-
Short review of Chapter 3 - basic functional analysis
-
Chapter 5 - ill conditioned linear systems: condition number, SVD, Tikhonov
regularization
-
Chapter 7 - matrix eigenvalue problems: estimates for eigenvalues, Jacobi
method, QR algorighm, Hessenberg matrices
-
Chapter 9 - numerical integration: quadrature based on interpolation, convergence
of quadrature formulas, orthogonal polynomials and Gaussian quadrature,
Romberg integration
-
Chapter 10 - initial value problems: existence and uniqueness, Euler method,
single step methods, multi step methods
-
Chapter 11 -boundary value problems: shooting methods, finite difference
methods, the Riesz and Lax-Milgram theorems, weak solutions and Sobolev
spaces, the finite element method
How to study: Question and distrust all material in the book
and in the class, discover things for yourself, do not stop until you understand.
Study in a group. Bring your questions to the class and to your study group.
Do not just "go over the material," that is a waste of time. Solve as many
problems from the text as possible, the homeworks can cover only a small
part of them.
Material covered and homeworks
Homework solutions are here.
8/20/01: introduction to Matlab, examples of linear systems (2.1)
8/22/01: introduction to Matlab, examples of linear systems (2.1)
8/27/01: review of Linear Algebra: vector spaces, basis
8/29/01: review of Linear Algebra: matrices
Homework 1 due 9/5/01: 2.10,2.11
9/5/01: Gauss elimination (2.2)
Homework 2 due 9/19/01: 2.1,2.2. For 2.2, create Matlab function x=elim(A,b) in file elim.m
that solves A*x=b for a square matrix A. Call the function elim from script in file go.m
that sets up and solves two systems of equations of dimension at least 5, and tests
the solution (A*x-b should be close to zero). Keep your programming simple, use for-end loops.
Put all files
in directory 5660f01/hw1 in your linus account. Make sure the permissions are set so that
others can read and list the directory.
9/10/01: Gauss elimination in matrix form and LR decomposition. Created lr.m
9/12/01: Gauss elimination for SPD matrices and Choleski decomposition.
9/17/01: QR decomposition
9/19/01: Least squares by QR decomposition. Homework 3 due 9/26/01. For the
a survey of the least squares and the homework see here.
NOTE: If you see a blank page when clicking on this link, this is a problem
some versions of Microsoft Internet Explorer have with PDF files.
(Thanks, Bill!) You can do any of the following:
1. Use another browser instead of Microsoft IE,
like Netscape 2. download and open the PDF file (right click, "Save target as",
"Open folder", click on the file)
3. clear IE cache before clicking on the link.
9/24/01 Ch. 3 - normed linear spaces
9/26/01 Ch. 3 - matrix norms, inner product spaces
10/3/01 Ch. 3 - Schur decomposition, eigenvalues Homework 4 due 10/10: 3.4, 3.5, 3.10, 3.11 Solution
10/8/01 Ch. 3 - Spectral norm
10/10/01 Ch. 3 - Completeness and Banach contraction theorem
10/15/01 Ch. 3 - projection (review from LA); best approximation; review for Test 1
10/17/01 Test 1
10/22/01 Ch. 3 - projections
10/24/01 Ch. 4 - Iterative methods (formulation of Jacobi, GS, SOR) Homework 5:
4.1 (Hint: just formulat the iteration matrix and compute its max norm).
4.2 (Write the program in Matlab from the definition of the methods, see file iter.m in
the Matlab area for matrix form. Put your files in directory 5660f01/hw5 in your account.)
11/5/01 Ch. 6 - Nonlinear iterations
11/7/01 Ch. 6 - Nonlinear iterations HW 6: 6.4, 6.12
solution
11/14/01 Ch. 6 - Fixed point and Newton method in n dimensions.
HW7 due 11/21 solution
11/19/01 Ch. 8 - Lagrange interpolation
11/21/01 Chebyshev interpolation ,
Ch. 8 - Hermite interpolation
HW8 due 11/28
Review for Test 1
You can expect the problems on the test to be picked from the following types of problems.
You can use books, notes, or calculators.
- Given a small square matrix, compute LR decomposition by Gauss elimination by hand
- Given a small square matrix, write one step of Gauss elimination as multiplication
of the matrix by a triangular matrix from the left
- Given a small square matrix, do Gauss elimination with partial pivoting by hand
- Given a small square matrix, do Choleski decomposition by hand
- Given a small square matrix, compute QR decomposition by hand
- Given a small square matrix, perform the first step of QR decomposition (i.e., find the first
Householder matrix)
- Given a matrix and the Householder matrices in QR decomposition, find Q.
- Find (and prove!) if a given function (e.g. sqrt(x(1))+sqrt(x(2)) is a norm or not
- Find (and prove!) if a given function (e.g., x(1)*y(2) + x(2)*y(1)i) is a scalar product or not
- Compute the 1-norm and infinity norm of a given matrix.
- Let ||.|| be vector norm and the induced matrix norm. What is ||I||? What is ||0||? Prove the answer form the definition of norm of linear operator.
- If ||Ax|| <= C ||x|| for all x, what can you say about ||A||? Why? (=I am asking for a proof)
- If ||Ax|| >= C ||x|| for some x, what can you say about ||A||? Why?
- If A is a linera operator and ||Ax_n|| >= C ||x_n|| for a sequence x_n, and lim C_n = infinity,
what can you say about ||A||? Why?
- given a set of vectors, perform Gram-Schmidt orthogonalization
- given an orthogonal system, complete it to an orthogonal basis (complete by linearly
independent vectors to a basis - the additional vectors will be easy to guess- then apply Gram-Schmidt orthogonalization)
- Find the Schur decomposition of a 2x2 matrix (calculate one eigenvalue from definition,
perform the steps in the proof of existence of the decomposition, verify answer)
- Find the Schur decomposition of a 3x3 matrix, where one eigenvalue is easy to guess (proceed
as in the proof)
- Given a small matrix, compute its spectral radius (if square) and spectral norm
- Decide if the linear operator given by a matrix is contraction
- homework problems and examples done in class
Review for Test 2
You can expect the problems on the test to be picked from the following types of problems.
(Under construction, more coming soon)
- decide if given matrix P is a projection (need to check if PP=P), orthogonal projection
(check in addition P'=P)
- given orthogonal vectors u1 u2 in R^3, compute the orthogonal projection onto the space
spanned by these vectors (use Corollary 3.54)
- If x_n = A x_{n-1} + b and A is given, decide if the iterations converge
(if some norm of A is less than one this is easy, otherwise you have to compute all
eigenvalues of A and get the spectral radius)
- decide if Gauss Seidel iterations for a given matrix converges (use
the criterion of diagonal dominance or matrix is symmetric and positive definite)
- given a matrix A size 3 by 3, vectors b and x0, do one step of the Gauss-Seidel
iterative method that is compute x1 (or SOR with a given omega, or Jacobi)
- given a function f, find a bound on the error of Lagrange interpolation on
3 nodes (also Hermite and Chebychev, also 2 nodes)
- write the interpolation polynomial for a small number of nodes for one of
above types, from the definition
- same for SOR
- write Newton's method for the nonlinear system cos x + sin y = 1 cos x + sin y =1
- given t_0=0 t_1=pi and y_0 y_1 find trigonometric interpolant (with sin cos)
- same but find polynomial of exp(it)
- given f(x)= cos(x) - x write Newton's method for f(x)=0 as x_{n+1} = F(x_n)
and compute using a calculator the first two iterations given x_0
- given x_i y_i i=0,1,2 find cubic spline such that s(x_i)=s(y_i), from definition
- given f on [0,1] find n so that the error of piecewise linear interplation of f on
equistant nodes x_0=0,...x_n=1 is less than given tolerance in the whole [0,1]
- same for piecewise cubic interpolation using Hermite interpolation on each subinterval
You can use books, notes, or calculators, but no laptops.
Topics covered and homeworks Spring 2002
1/23 - Ch. 3: normed spaces
1/28 - Ch. 3: inner product spaces. HW1 due 2/4: program Gram-Schmidt
orthogonalization. Create Matlab function [Q,C]=myorth(U). The input
vectors are columns of U, output are columns of Q, C
contains the multipliers used in the orthogonalization process.
Create a script go.m
to exercise it on few examples. Verify that the result is correct using
an equation relating Q,C, and U. Put file myorth.m and go.m in directory
5661/hw1 in your math accout. Make sure the files and all directories above
them are world readable (chmod og+rx filename) so that I can get to them.
1/30 - Ch. 3: eigenvalues and eigenvectors, Hermitean matrices
2/4 - Ch. 3: best approximation, orthogonal projection, normal equations,
least squares. Ch. 5 - introductory example, least squares approximation by polynomials. By popular demand files lsapprox.m (for the in-class demo)
and also jmbasis.m (to get orthogonal basis of column space of matrix by
jmqr.m which is qr decomposition adapted from my demos in Fall) are
in the Matlab area.
HW2 due 2/11: Apply Gram-Schmidt orthogonalization to functions 1, x, x^2,...
in the L^2(-1,1) inner product. Compute by hand the first four polynomials
and graph them in Matlab, turn in hardcopy. Suggestion: use Matlab
functions polyval, inline, fplot. Print in Matlab to a file using the
pull-down menu on the figure window. When running without graphics,
you may use the fplot command to create the figure (in memory, not
visible) then print it to a file by a command like
print -dps myfigure.ps (to get a postscript file, use help print in matlab
to see how to get files for other types of printers)
2/6 - Ch. 3 - Neumann series, Ch. 5 - condition number
2/11 - Ch. 5 SVD; hw 3 due 2/17: 5.13, 5.15
2/13 - Ch. 5 minimum norm least squares solution and
Moore-Penrose pseudoinverse, pseudoinverse
demonstration
2/17 - Ch. 5 - spectral cutoff and Tikhonov regularization
Homework 4 due 2/25/02
2/19 - Ch. 5 Tikhonov regularization (conclusion), example:
image reconstruction from projections
2/25 image reconstruction from projections (conclusion), regularization wrapup;
Ch. 7: Rayleigh quotient (Thm 7.3, 7.4), Frobenius norm (sec 7.3)
2/27
Frobenius norm (sec 7.3), Jacobi method
3/4 Ch. 7 - Jacobi method (conclusion), Gerschgorin estimates, qr demo
3/6 - Review of Ch. 6 and covered part of Ch. 7
Review questions for Test 1
- Given a (small) matrix A, right hand side y and alpha, compute approximate
solution of Ax=y by Tikhonov regularization with given alpha.
- Given matrix A in the form of SVD, vector y and tolerance delta compute
approximate solution by spectral cutoff
- If A is real symmetric with known eigenvalues and orthonormal eigenvectors what is SVD
of A. Justify from definition of SVD.
- Can SVD be computed by a finite number of algebraic operations and roots?
- Compute SVD of a given matrix size m by n (m+n <= 4) following through the proof
of Thm 5.4
- For A given as SVD write A as sum of rank one matrices.
- For A given as SVD write pseudoinverse of A as sum of rank one matrices
- For a given real symmetric A localize the eigenvalues of A from the sum of squares of
off-diagonal entries
- If x is eigenvector of A show that the derivative of the Rayleigh quotiend in the
direction x is zero
- If the matrices U and V in SVD of A are same, what can you say about A?
- Show that the Rayleigh quotient of real symmetric matrix is between minimal and
maximal eigenvalue
- Perform one step of classical Jacobi method (use formulas in Lemma 7.13 to find the
rotation angle phi) for a matrix of size 3
- Given matrix A compute N(A) after one step of Jacobi method with given j k
(do not perform the step)
- Why the Jacobi method does not change eigenvalues of A?
- Given matrix A localize its eigenvalues using Gerschgorin theorem
3/11 - Test 1
3/13 - power method, QR method (simple case only), Hessenberg form (demo only)
3/18 - Ch. 9 - quadrature: interpolatory quadrature
3/20 - Ch. 9 - composite rules, Gaussian quadrature
4/1 - Ch. 9 - Gaussian quadrature (conclusion), convergence of quadrature rules
Homework 5 due April 8: Develop two-point Gaussian quadrature rule
for integrals on (0,1) of functions of the form |log x| f(x) . Hint: use
Gram-Schmidt orthogonalization to
find the orthogonal polynomials on (0,1) with respect to weight function |log x|,
then find the quadrature nodes as roots of the orthogonal polynomial of order 2,
finally find the weights
from the requirement that the quadrature is exact for f(x) polynomial of order up to 1.
solution
4/3 - Ch. 10: initial value problesm. Picard-Lindeloef theorem,
Picard iterations
4/8 - Ch. 10: Euler method, improved Euler method, Single-step methods
4/10 - Ch. 10: local error analysis
4/15 - Ch. 10: global error analysis
hw6: due 4/22: Apply Euler method, trapezoidal method, Heun method, and ode45 from Matlab to
the IVP u'=-100*(u-cos(x)), u(0)=0. Plot solution for h=0.1 and sequence decreasing h by halving.
Find this way numerically h that makes error < 1e-6 at x=10. Extra credit: same for u(0)=1
4/17 - Ch. 10 - multipoint methods
4/22 - Ch. 10 - multipoint methods
4/24 - Ch. 11 - Boundary value problems: shooting methods
4/29 - Ch. 11 - Boundary value problems: difference methods
hw7 due 5/6/02: Reproduce results in Table 11.1; turn in all as hardcopy
5/1/002: Ch. 11: error estimates for difference methods
5/6/02: Ch. 11 - Bouldary value problem: variational methods (overview)
5/8/02: review
Review questions for the final:
- You are given symmetric matrix A such that the eigenvalues of A are easy to
comput and vector x_0. Define x_{n+1} =
A x_n. What can you say about the limit of the sequence a_n = (x_n'*A*x_n)/(x_n'(x_n)? How is this method called, what is its practical form?
- Approximate the integral of sin(x) from 0 to 1 by the composite trapezoidal rule
using the nodes 0, 1/2, 1 and give an error bound.
- Same as above using Gauss quadrature
with two nodes. Derive the Gaussian quadrature nodes and weights in detail, use formula for
error from the book.
- Gaussian quadrature on 10 nodes is exact for polynomials of what order?
- Write the equation u''=u^2 , u(0)=1, u'(0)=0, as a system
and perform one step of the Euler method with h=0.1.
- Perform one step of the Euler method for u'=u^2, u(0)=1, with h=0.1,
and estimate the error at x=0.1
- Perform the first Picard iteration for the differential equation u'=u^2,
u(0)=1, starting from the initial approximation u(x)=1
- Perform the first iteration of the two-point Adams-Bashford method.
What is the consistency order of the method? Write the definition of
consistency in this case.
- Given a multi-step method (for example, (10.23) or (10.26)), use
Taylor expansion of the solution to find its consistency order.
- Find if the method is stable.
- Given differential equation u''=-u-1, u(0)=u(1)=0,
set up the system of nonlinear algebraic
equations to be solved in the shooting method (solve the initial
value problems analytically)
- Summarize the steps in the bound of the driescretization error for the
finite difference solution of u''=1, u(0)=u(1)=0
- write the finite difference discretization of the above equation.
- use integration by parts to show that u''=0, u(0)=u(1)=0, has
only trivial solution.
Last updated 5/1/02