• Steepest Descent direction
• Newton's method for solving equations
• Newton-Like methods
• Q-convergence rates
• linear, superlinear, and quadratic convergence
• Newton's method for minimization
• The rate of convergence of Newton's method
• Matrix secant equation
• Broyden's method (both direct and inverse updates)
• the BFGS update (both direct and inverse updates)

• Gaussian elimination matrices
• upper and lower triangular matrices
• LU factorization
• Cholesky Factorization
• unitary transformations
• Householder reflections
• QR Factorization
• orthogonal projections

• Q-conjugacy
• Show Q-conjugacy implies linear independence
• conjugate direction methods
• Expanding subspace theorem
• the conjugate gradient algorithm (CGA)
• the Conjugate Gradient Theorem
• the non-quadratic CG algorithm
• the Fletcher-Reeves and Polak-Ribiere formula

• Steepest descent
• Newton's method for equations and minimization
• Broyden's method
• BFGS

• the LU, Cholesky, and QR factorizations

• Q-conjugacy
• the conjugate gradient algorithm

Skills to Master:

• Implementing Broyden and BFGS in Matlab
• Checking Q-cojugacy
• write a Matlab routing to implement the CGA

Quiz:

• The quiz will consist of 2 questions. The first question will be related to the vocabulary words from the notes on Search Direction through Newton's method for equation solving and minimization only, pages 1-6 of the notes (i.e. no other vocabulary words will be asked). In the second question you will be asked to solve a problem similar to one of the problems on Problem Set 5 and the topic for problem 5 on the midterm exam.