Course Meeting Times
Lectures: 2 sessions/week, 1.5 hours/session
Prerequisites
18.06 Linear Algebra and one of 18.100A Real Analysis, 18.100B Real Analysis, or 18.100Q Real Analysis.
Course Description
This course offers a unified analytical and computational approach to nonlinear optimization problems. Unconstrained optimization methods include gradient, conjugate direction, Newton, sub-gradient, and first-order methods. Constrained optimization methods include feasible directions, projection, interior point methods, and Lagrange multiplier methods. The curriculum covers convex analysis, Lagrangian relaxation, and nondifferentiable optimization, as well as applications in integer programming. It provides a comprehensive treatment of optimality conditions and Lagrange multipliers. The course also utilizes a geometric approach to duality theory. Finally, applications are drawn from control, communications, machine learning, and resource allocation problems.
This course introduces students to the fundamentals of nonlinear optimization theory and algorithms. When applicable, emphasis is put on modern applications, especially within machine learning and its sub-branches.
The course is roughly divided into three interconnected parts:
- Part I will focus on the characterization of optimal points from an analytic point of view. In particular,
we will develop analytic techniques to find and certify optimal points in convex and nonconvex problems. - Part II will focus on gradient-based optimization methods in convex and nonconvex problems.
- Part III will focus on preconditioned algorithms, second-order algorithms, or more generally algorithms that exploit the geometry of the objective function beyond its mere gradient.
Grading
| Activity | Percentage |
|---|---|
| Homework | 40% |
| Midterm | 30% |
| Final Exam | 30% |
Collaboration Policy
We encourage working together whenever possible: in the tutorials, problem sets, and general discussion of the material and assignments. Keep in mind, however, that for the problem sets the solutions you hand in should reflect your own understanding of the class material, and should be written solely by you. It is not acceptable to copy a solution that somebody else has written.
Late Policy
You have seven late days to use throughout the semester, no questions asked. Late days used for each homework are rounded up to the unit.
Policy on Language Models
The goal of the course is for you to understand the material. Use of LLMs towards that direction is allowed. We are going to assume that everyone is reasonable.
