Books on Optimization for Machine Learning-2

swapneel-panda-419bc751 · 30 June 2021 15:35

Numerical Optimization

This book was written by Jorge Nocedal and Stephen Wright and was published in 2006.

Numerical Optimization

This book is focused on the math and theory of the optimization algorithms presented and does cover many of the foundational techniques used by common machine learning algorithms. It may be a little too heavy for the average practitioner.

The book is intended as a textbook for graduate students in mathematical subjects.

We intend that this book will be used in graduate-level courses in optimization, as offered in engineering, operations research, computer science, and mathematics departments.

— Page xviii, Numerical Optimization, 2006.

Even though it is highly mathematical, the descriptions of the algorithms are precise and may provide a useful alternative description to complement the other books listed.

The complete table of contents for the book is listed below.

Chapter 01: Introduction
Chapter 02: Fundamentals of Unconstrained Optimization
Chapter 03: Line Search Methods
Chapter 04: Trust-Region Methods
Chapter 05: Conjugate Gradient Methods
Chapter 06: Quasi-Newton Methods
Chapter 07: Large-Scale Unconstrained Optimization
Chapter 08: Calculating Derivatives
Chapter 09: Derivative-Free Optimization
Chapter 10: Least-Squares Problems
Chapter 11: Nonlinear Equations
Chapter 12: Theory of Constrained Optimization
Chapter 13: Linear Programming: The Simplex Method
Chapter 14: Linear Programming: Interior-Point Methods
Chapter 15: Fundamentals of Algorithms for Nonlinear Constrained Optimization
Chapter 16: Quadratic Programming
Chapter 17: Penalty and Augmented Lagrangian Methods
Chapter 18: Sequential Quadratic Programming
Chapter 19: Interior-Point Methods for Nonlinear Programming

It’s a solid textbook on optimization.