Learning Theory from First Principles

A comprehensive and cutting-edge introduction to the foundations and modern applications of learning theory. Research has exploded in the field of machine learning resulting in complex mathematical arguments that are hard to grasp for new comers. . In this accessible textbook, Francis Bach presents the foundations and latest advances of learning theory for graduate students as well as researchers who want to acquire a basic mathematical understanding of the most widely used machine learning architectures. Taking the position that learning theory does not exist outside of algorithms that can be run in practice, this book focuses on the theoretical analysis of learning algorithms as it relates to their practical performance. Bach provides the simplest formulations that can be derived from first principles, constructing mathematically rigorous results and proofs without overwhelming students. Provides a balanced and unified treatment of most prevalent machine learning methods Emphasizes practical application and features only commonly used algorithmic frameworks Covers modern topics not found in existing texts, such as overparameterized models and structured prediction Integrates coverage of statistical theory, optimization theory, and approximation theory Focuses on adaptivity, allowing distinctions between various learning techniques Hands-on experiments, illustrative examples, and accompanying code link theoretical guarantees to practical behaviors