Reading Assignment:

• Nonlinear Programming Notes: The Conjugate Gradient Algorithm: Due Wednesday, February 27.
• Nonlinear Programming Notes: Optimality Conditions for Constrained Problems: Due Wednesday, March 6.
• Nonlinear Programming Notes: The Gradient Projection Algorithm: Due Wednesday, March 13.

Homework Assignment:

• Problem Set 7. Due Monday, March 4.
• Problem Set 8. Due Monday, March 11.

Vocabulary Words

• The Conjugate Gradient Algorithm
  • Q-conjugacy
  • Show Q-conjugacy implies linear independence
  • conjugate direction methods
  • Expanding subspace theorem
  • the conjugate gradient algorithm (CGA)
  • the Conjugate Gradient Theorem
  • the non-quadratic CG algorithm
  • the Fletcher-Reeves and Polak-Ribiere formula
• Optimality Conditions for Constrained Problems
  • Tangent directions
  • Basic Constrained 1st-Order Optimality Conditions
  • Regularity
  • the Lagrangian and Lagrange multipliers
  • Constrained 1st-Order Optimality Conditions
  • Karush-Huhn-Tucker (KKT) conditions
  • Constrained 2nd-Order Necessary and Sufficient Conditions
  • 1st-Order Necesaary and Sufficient Conditions in Convex Optimization
• The gradient Projection Algorithm
  • the projection onto a convex set and the projection mapping
  • Projection Theorem for Convex Sets
  • the normal cone for convex sets
  • 1st-order optimality and the normal cone
  • projections and the normal cone
  • gradient projection search direction
  • the basic gradient projection algorithm
  • convergence of the gradient projection algorithm
  • projections onto boxes
• Quadratic Programming
  • what is a quadratic program
  • QP standard form
  • KKT conditions for QPs in standard form

Key Concepts:

• The Conjugate Gradient Algorithm
  • Q-conjugacy
  • the conjugate gradient algorithm
• Optimality Conditions for Constrained Problems
  • regularity
  • the linear independence condition
  • the KKT conditions
  • the Lagrangian
  • convex duality
• Gradient Projection Algorithm
  • projections onto convex sets, especially boxes
  • the gradient projection search direction
  • the gradient projection algorithm

Skills to Master: