Py-BOBYQA: Derivative-Free Optimizer for Bound-Constrained Minimization

Release: 1.4.1

Date: 11 April 2024

Author: Lindon Roberts

Py-BOBYQA is a flexible package for finding local solutions to nonlinear, nonconvex minimization problems (with optional bound constraints), without requiring any derivatives of the objective. Py-BOBYQA is a Python implementation of the BOBYQA solver by Powell (documentation here). It is particularly useful when evaluations of the objective function are expensive and/or noisy.

That is, Py-BOBYQA solves

\[\begin{split}\min_{x\in\mathbb{R}^n} &\quad f(x)\\ \text{s.t.} &\quad a \leq x \leq b\end{split}\]

The upper and lower bounds on the variables are non-relaxable (i.e. Py-BOBYQA will never ask to evaluate a point outside the bounds).

Full details of the Py-BOBYQA algorithm are given in our papers:

1. Coralia Cartis, Jan Fiala, Benjamin Marteau and Lindon Roberts, Improving the Flexibility and Robustness of Model-Based Derivative-Free Optimization Solvers, ACM Transactions on Mathematical Software, 45:3 (2019), pp. 32:1-32:41 [preprint]

2. Coralia Cartis, Lindon Roberts and Oliver Sheridan-Methven, Escaping local minima with derivative-free methods: a numerical investigation, Optimization, 71:8 (2022), pp. 2343-2373. [arXiv preprint: 1812.11343]

Please cite [1] when using Py-BOBYQA for local optimization, and [1,2] when using Py-BOBYQA’s global optimization heuristic functionality.

If you are interested in solving least-squares minimization problems, you may wish to try DFO-LS, which has the same features as Py-BOBYQA (plus some more), and exploits the least-squares problem structure, so performs better on such problems.

Since v1.1, Py-BOBYQA has a heuristic for global optimization (see Using Py-BOBYQA for details). As this is a heuristic, there are no guarantees it will find a global minimum, but it is more likely to escape local minima if there are better values nearby.

Py-BOBYQA is released under the GNU General Public License. Please contact NAG for alternative licensing.

Contents:

• Installing Py-BOBYQA
  • Requirements
  • Installation using pip
  • Manual installation
  • Testing
  • Uninstallation
• Overview
  • When to use Py-BOBYQA
  • Details of the Py-BOBYQA Algorithm
  • References
• Using Py-BOBYQA
  • Nonlinear Minimization
  • How to use Py-BOBYQA
  • Optional Arguments
  • A Simple Example
  • Adding Bounds and More Output
  • Example: Noisy Objective Evaluation
  • Example: Global Optimization
  • References
• Advanced Usage
  • General Algorithm Parameters
  • Logging and Output
  • Initialization of Points
  • Trust Region Management
  • Termination on Small Objective Value
  • Termination on Slow Progress
  • Stochastic Noise Information
  • Interpolation Management
  • Multiple Restarts
  • References
• Diagnostic Information
  • Current Iterate
  • Trust Region
  • Model Interpolation
  • Iteration Count
  • Algorithm Progress
• Version History
  • Version 1.0 (6 Feb 2018)
  • Version 1.0.1 (20 Feb 2018)
  • Version 1.0.2 (20 Jun 2018)
  • Version 1.1 (24 Dec 2018)
  • Version 1.1.1 (5 Apr 2019)
  • Version 1.2 (25 Feb 2020)
  • Version 1.3 (14 Apr 2021)
  • Version 1.4 (16 May 2023)
  • Version 1.4.1 (11 Apr 2024)

Acknowledgements

This software was developed under the supervision of Coralia Cartis, and was supported by the EPSRC Centre For Doctoral Training in Industrially Focused Mathematical Modelling (EP/L015803/1) in collaboration with the Numerical Algorithms Group.