Py-BOBYQA: Derivative-Free Optimizer for Bound-Constrained Minimization

Release: 1.5.0

Date: 16 September 2024

Author: Lindon Roberts

Py-BOBYQA is a flexible package for finding local solutions to nonlinear, nonconvex minimization problems (with optional bound and other convex constraints), without requiring any derivatives of the objective. Py-BOBYQA is a Python implementation of the BOBYQA solver by Powell (documentation here). It is particularly useful when evaluations of the objective function are expensive and/or noisy.

That is, Py-BOBYQA solves

\[\begin{split}\min_{x\in\mathbb{R}^n} &\quad f(x)\\ \text{s.t.} &\quad a \leq x \leq b \\ &\quad x \in C := C_1 \cap \cdots \cap C_n, \quad \text{all $C_i$ convex}\end{split}\]

If provided, the constraints the variables are non-relaxable (i.e. Py-BOBYQA will never ask to evaluate a point outside the bounds), although the general \(x \in C\) constraint may be slightly violated from rounding errors.

Full details of the Py-BOBYQA algorithm are given in our papers:

1. Coralia Cartis, Jan Fiala, Benjamin Marteau and Lindon Roberts, Improving the Flexibility and Robustness of Model-Based Derivative-Free Optimization Solvers, ACM Transactions on Mathematical Software, 45:3 (2019), pp. 32:1-32:41 [preprint]

2. Coralia Cartis, Lindon Roberts and Oliver Sheridan-Methven, Escaping local minima with derivative-free methods: a numerical investigation, Optimization, 71:8 (2022), pp. 2343-2373. [arXiv preprint: 1812.11343]

3. Lindon Roberts, Model Construction for Convex-Constrained Derivative-Free Optimization, arXiv preprint arXiv:2403.14960 (2024).

Please cite [1] when using Py-BOBYQA for local optimization, [1,2] when using Py-BOBYQA’s global optimization heuristic functionality, and [1,3] if using the general convex constraints \(x \in C\) functionality.

If you are interested in solving least-squares minimization problems, you may wish to try DFO-LS, which has the same features as Py-BOBYQA (plus some more), and exploits the least-squares problem structure, so performs better on such problems.

Since v1.1, Py-BOBYQA has a heuristic for global optimization (see Using Py-BOBYQA for details). As this is a heuristic, there are no guarantees it will find a global minimum, but it is more likely to escape local minima if there are better values nearby.

Py-BOBYQA is released under the GNU General Public License. Please contact NAG for alternative licensing.

Contents:

• Installing Py-BOBYQA
  • Requirements
  • Installation using pip
  • Manual installation
  • Testing
  • Uninstallation
• Overview
  • When to use Py-BOBYQA
  • Details of the Py-BOBYQA Algorithm
  • References
• Using Py-BOBYQA
  • Nonlinear Minimization
  • How to use Py-BOBYQA
  • Optional Arguments
  • A Simple Example
  • Adding Bounds and More Output
  • Adding General Convex Constraints
  • Example: Noisy Objective Evaluation
  • Example: Global Optimization
  • References
• Advanced Usage
  • General Algorithm Parameters
  • Logging and Output
  • Initialization of Points
  • Trust Region Management
  • Termination on Small Objective Value
  • Termination on Slow Progress
  • Stochastic Noise Information
  • Interpolation Management
  • Multiple Restarts
  • General Convex Constraints
  • References
• Diagnostic Information
  • Current Iterate
  • Trust Region
  • Model Interpolation
  • Iteration Count
  • Algorithm Progress
• Version History
  • Version 1.0 (6 Feb 2018)
  • Version 1.0.1 (20 Feb 2018)
  • Version 1.0.2 (20 Jun 2018)
  • Version 1.1 (24 Dec 2018)
  • Version 1.1.1 (5 Apr 2019)
  • Version 1.2 (25 Feb 2020)
  • Version 1.3 (14 Apr 2021)
  • Version 1.4 (16 May 2023)
  • Version 1.4.1 (11 Apr 2024)
  • Version 1.5.0 (16 Sep 2024)

Acknowledgements

This software was initially developed under the supervision of Coralia Cartis, and was supported by the EPSRC Centre For Doctoral Training in Industrially Focused Mathematical Modelling (EP/L015803/1) in collaboration with the Numerical Algorithms Group. Development of Py-BOBYQA has also been supported by the Australian Research Council (DE240100006).