Advanced Usage

This section describes different optional user parameters available in Py-BOBYQA.

In the last section (Using Py-BOBYQA), we introduced pybobyqa.solve(), which has the optional input user_params. This is a Python dictionary of user parameters. We will now go through the settings which can be changed in this way. More details are available in the paper [CFMR2018].

The default values, used if no override is given, in some cases vary depending on whether objfun has stochastic noise; that is, whether evaluating objfun(x) several times at the same x gives the same result or not. Whether or not this is the case is determined by the objfun_has_noise input to pybobyqa.solve() (and not by inspecting objfun, for instance). Similarly, the default values depend on the input flag seek_global_minimum, i.e. if a global minimum is desired.

General Algorithm Parameters

general.rounding_error_constant - Internally, all interpolation points are stored with respect to a base point \(x_b\); that is, we store \(\{y_t-x_b\}\), which reduces the risk of roundoff errors. We shift \(x_b\) to \(x_k\) when \(\|s_k\| \leq \text{const}\|x_k-x_b\|\), where ‘const’ is this parameter. Default is 0.1.
general.safety_step_thresh - Threshold for when to call the safety step, \(\|s_k\| \leq \gamma_S \rho_k\). Default is \(\gamma_S =0.5\).
general.check_objfun_for_overflow - Whether to cap the value of \(r_i(x)\) when they are large enough that an OverflowError will be encountered when trying to evaluate \(f(x)\). Default is True.

Logging and Output

logging.n_to_print_whole_x_vector - If printing all function evaluations to screen/log file, the maximum len(x) for which the full vector x should be printed also. Default is 6.
logging.save_diagnostic_info - Flag so save diagnostic information at each iteration. Default is False.
logging.save_poisedness - If saving diagnostic information, whether to include the \(\Lambda\)-poisedness of \(Y_k\) in the diagnostic information. This is the most computationally expensive piece of diagnostic information. Default is True.
logging.save_xk - If saving diagnostic information, whether to include the full vector \(x_k\). Default is False.

Initialization of Points

init.random_initial_directions - Build the initial interpolation set using random directions (as opposed to coordinate directions). Default is True. Not used if general convex constraints provided.
init.random_directions_make_orthogonal - If building initial interpolation set with random directions, whether or not these should be orthogonalized. Default is True. Not used if general convex constraints provided.
init.run_in_parallel - If using random directions, whether or not to ask for all objfun to be evaluated at all points without any intermediate processing. Default is False. Not used if general convex constraints provided.

Trust Region Management

tr_radius.eta1 - Threshold for unsuccessful trust region iteration, \(\eta_1\). Default is 0.1.
tr_radius.eta2 - Threshold for very successful trust region iteration, \(\eta_2\). Default is 0.7.
tr_radius.gamma_dec - Ratio to decrease \(\Delta_k\) in unsuccessful iteration, \(\gamma_{dec}\). Default is 0.5 for smooth problems or 0.98 for noisy problems (i.e. objfun_has_noise = True).
tr_radius.gamma_inc - Ratio to increase \(\Delta_k\) in very successful iterations, \(\gamma_{inc}\). Default is 2.
tr_radius.gamma_inc_overline - Ratio of \(\|s_k\|\) to increase \(\Delta_k\) by in very successful iterations, \(\overline{\gamma}_{inc}\). Default is 4.
tr_radius.alpha1 - Ratio to decrease \(\rho_k\) by when it is reduced, \(\alpha_1\). Default is 0.1 for smooth problems or 0.9 for noisy problems (i.e. objfun_has_noise = True).
tr_radius.alpha2 - Ratio of \(\rho_k\) to decrease \(\Delta_k\) by when \(\rho_k\) is reduced, \(\alpha_2\). Default is 0.5 for smooth problems or 0.95 for noisy problems (i.e. objfun_has_noise = True).

Termination on Small Objective Value

model.abs_tol - Tolerance on \(f(x_k)\); quit if \(f(x_k)\) is below this value. Default is \(-10^{20}\).

Termination on Slow Progress

slow.history_for_slow - History used to determine whether the current iteration is ‘slow’. Default is 5.
slow.thresh_for_slow - Threshold for objective decrease used to determine whether the current iteration is ‘slow’. Default is \(10^{-8}\).
slow.max_slow_iters - Number of consecutive slow successful iterations before termination (or restart). Default is 20*len(x0).

Stochastic Noise Information

noise.quit_on_noise_level - Flag to quit (or restart) if all \(f(y_t)\) are within noise level of \(f(x_k)\). Default is False for smooth problems or True for noisy problems.
noise.scale_factor_for_quit - Factor of noise level to use in termination criterion. Default is 1.
noise.multiplicative_noise_level - Multiplicative noise level in \(f\). Can only specify one of multiplicative or additive noise levels. Default is None.
noise.additive_noise_level - Additive noise level in \(f\). Can only specify one of multiplicative or additive noise levels. Default is None.

Interpolation Management

interpolation.precondition - whether or not to scale the interpolation linear system to improve conditioning. Default is True.
interpolation.minimum_change_hessian - whether to solve the underdetermined quadratic interpolation problem by minimizing the Frobenius norm of the Hessian, or change in Hessian. Default is True.

Multiple Restarts

restarts.use_restarts - Whether to do restarts when \(\rho_k\) reaches \(\rho_{end}\), or (optionally) when all points are within noise level of \(f(x_k)\). Default is False for smooth problems or True for noisy problems or when seeking a global minimum.
restarts.max_unsuccessful_restarts - Maximum number of consecutive unsuccessful restarts allowed (i.e.~restarts which did not reduce the objective further). Default is 10.
restarts.max_unsuccessful_restarts_total - Maximum number of total unsuccessful restarts allowed. Default is 20 when seeking a global minimum, otherwise it is maxfun (i.e.~not restricted).
restarts.rhobeg_scale_after_unsuccessful_restart - Factor to increase \(\rho_{beg}\) by after unsuccessful restarts. Default is 1.1 when seeking a global minimum, otherwise it is 1.
restarts.rhoend_scale - Factor to reduce \(\rho_{end}\) by with each restart. Default is 1.
restarts.use_soft_restarts - Whether to use soft or hard restarts. Default is True.
restarts.soft.num_geom_steps - For soft restarts, the number of points to move. Default is 3.
restarts.soft.move_xk - For soft restarts, whether to preserve \(x_k\), or move it to the best new point evaluated. Default is True.
restarts.hard.use_old_fk - If using hard restarts, whether or not to recycle the objective value at the best iterate found when performing a restart. This saves one objective evaluation. Default is True.
restarts.soft.max_fake_successful_steps - The maximum number of successful steps in a given run where the new (smaller) objective value is larger than the best value found in a previous run. Default is maxfun, the input to pybobyqa.solve().
restarts.auto_detect - Whether or not to automatically determine when to restart. This is an extra condition, and restarts can still be triggered by small trust region radius, etc. Default is True.
restarts.auto_detect.history - How many iterations of data on model changes and trust region radii to store. There are two criteria used: trust region radius decreases (no increases over the history, more decreases than no changes), and change in model Jacobian (consistently increasing trend as measured by slope and correlation coefficient of line of best fit). Default is 30.
restarts.auto_detect.min_chg_model_slope - Minimum rate of increase of \(\log(\|g_k-g_{k-1}\|)\) and \(\log(\|H_k-H_{k-1}\|_F)\) over the past iterations to cause a restart. Default is 0.015.
restarts.auto_detect.min_correl - Minimum correlation of the data sets \((k, \log(\|g_k-g_{k-1}\|))\) and \((k, \log(\|H_k-H_{k-1}\|_F))\) required to cause a restart. Default is 0.1.

General Convex Constraints

projections.dykstra.d_tol - termination tolerance for Dykstra’s algorithm for computing the projection onto the intersection of all convex constraints. Default is \(10^{-10}\).
projections.dykstra.max_iters - maximum iterations of Dykstra’s algorithm for computing the projection onto the intersection of all convex constraints. Default is 100.
projections.feasible_tol - tolerance for checking feasibility of initial points with respect to general convex constraints. Default is \(10^{-10}\).
projections.pgd_tol - termination tolerance for trust-region and geometry-improving subproblems. Default is \(10^{-8}\).

References

[CFMR2018]

Coralia Cartis, Jan Fiala, Benjamin Marteau and Lindon Roberts, Improving the Flexibility and Robustness of Model-Based Derivative-Free Optimization Solvers, ACM Transactions on Mathematical Software, 45:3 (2019), pp. 32:1-32:41 [preprint]