This is a monograph on Bayesian optimization that was published in early 2023 by Cambridge University Press.

The book aims to provide a self-contained and comprehensive introduction to Bayesian optimization, starting “from scratch” and carefully developing all the key ideas along the way. The intended audience is graduate students and researchers in machine learning, statistics, and related fields. However, I also hope that practitioners and researchers from more distant fields will find some utility here.

The book is divided into three main parts, covering:

• theoretical and practical aspects of Gaussian process modeling,
• the Bayesian approach to sequential decision making, and
• the realization of practical and effective optimization policies.

A few additional topics are also covered:

• an overview of theoretical convergence results,
• a survey of notable extensions,
• a comprehensive history of Bayesian optimization, and
• an annotated bibliography of applications.

You can order a copy from Cambridge University Press or from Amazon.

The book will remain freely available on this website for personal use.

Table of Contents

1. Introduction
2. Gaussian Processes
3. Modeling with Gaussian Processes
4. Model Assessment, Selection, and Averaging
5. Decision Theory for Optimization
6. Utility Functions for Optimization
7. Common Bayesian Optimization Policies
8. Computing Policies with Gaussian Processes
9. Implementation
10. Theoretical Analysis
11. Extensions and Related Settings
12. A Brief History of Bayesian Optimization

Download

You may download the book in several slightly different formats below. The 8”x10” version shares the same page size as what will appear in print and is best for online viewing. US letter and A4 versions are also provided for ease of printing.

• Download 8"x10"
• Download US Letter
• Download A4

Feedback/Errata

I welcome feedback, errata, etc. Please feel free to report an issue on the associated GitHub repository.

A list of errata is available here.

Cite

@book{garnett_bayesoptbook_2023,
  author    = {Garnett, Roman},
  title     = {{Bayesian Optimization}},
  year      = {2023},
  publisher = {Cambridge University Press}
}