ScipyOptimizer

ScipyOptimizer#

class braintools.optim.ScipyOptimizer(loss_fun, bounds, method=None, constraints=(), tol=None, callback=None, options=None)#

SciPy-based optimizer with dict/sequence bounds compatible with Nevergrad.

This optimizer accepts the same bounds structure as braintools.optim.NevergradOptimizer and a loss function with matching signature:

  • If bounds is a sequence of (min, max) pairs, loss_fun is called positionally as loss_fun(*params).

  • If bounds is a dict mapping names to (min, max) pairs, loss_fun is called by keyword as loss_fun(**params).

Bounds can be scalars or arrays, and may optionally be provided as brainunit.Quantity. Internally, SciPy works on the numeric mantissas while your loss_fun receives values in the same structure as the bounds (without units).

Parameters:
  • loss_fun (Callable) – Objective function returning a scalar (0-D array-like). Its signature must match the structure of bounds as described above.

  • bounds (Sequence | Dict[str, Any]) – Search space bounds, as in Nevergrad: each value is (min, max) where elements are scalars or arrays, optionally u.Quantity. All elements in a pair must share shape, and units (if any) must be compatible. For dict bounds, names define keyword arguments to loss_fun.

  • method (str | None) – SciPy method (e.g., 'L-BFGS-B', 'TNC', 'SLSQP', 'Powell'). If omitted, SciPy selects a default based on constraints/bounds.

  • constraints – Passed through to scipy.optimize.minimize.

  • tol – Passed through to scipy.optimize.minimize.

  • callback – Passed through to scipy.optimize.minimize.

  • options – Passed through to scipy.optimize.minimize.

Notes

  • This wrapper flattens parameters to a 1-D vector for SciPy and unflattens results back to the same structure found in bounds.

  • Gradients are computed via JAX auto-differentiation, but only for gradient-based methods. Gradient-free methods ('Nelder-Mead', 'Powell', 'COBYLA', 'COBYQA') skip the Jacobian entirely.

  • Everything SciPy operates on (the parameter vector, the Jacobian and the bounds) is cast to float64 so that the 'TNC' and 'SLSQP' kernels work despite JAX defaulting to float32. The loss and gradient themselves are still computed at JAX’s precision; SciPy’s line searches assume double precision and can stop early (“Linear search failed”) on float32 values. Enable jax.config.update('jax_enable_x64', True) (after importing braintools) for tight fits.

  • With brainunit quantities in bounds, loss_fun receives plain numbers in the unit of each lower bound and res.x carries no units. Starting points are drawn with NumPy’s global random generator.

  • Constraint functions receive SciPy’s flat parameter vector: list bounds concatenated in order, dict bounds in sorted-key order.

  • This is a black-box optimizer: it implements minimize() only and does not honor the State-based register_trainable_weights() / update() contract of the Optimizer base (those raise NotImplementedError).

Examples

Minimize a simple quadratic with tuple bounds:

>>> import numpy as np
>>> from braintools.optim import ScipyOptimizer
>>> np.random.seed(0)  # starting points come from NumPy's global generator
>>> def loss(x, y):
...     return (x - 1.0)**2 + (y + 2.0)**2
>>> bounds = [(-5.0, 5.0), (-3.0, 3.0)]
>>> opt = ScipyOptimizer(loss_fun=loss, bounds=bounds, method='L-BFGS-B')
>>> res = opt.minimize(n_iter=3)
>>> isinstance(res.x, (list, tuple))  # same structure as bounds
True
>>> [round(float(v), 3) for v in res.x]
[1.0, -2.0]

With named parameters using dict bounds:

>>> def loss(**p):
...     return (p['a'] - 1.0)**2 + (p['b'] + 2.0)**2
>>> bounds = {"a": (-5.0, 5.0), "b": (-3.0, 3.0)}
>>> opt = ScipyOptimizer(loss_fun=loss, bounds=bounds, method='TNC')
>>> res = opt.minimize(n_iter=2)
>>> isinstance(res.x, dict)
True
>>> {k: round(float(v), 2) for k, v in res.x.items()}
{'a': 1.0, 'b': -2.0}
minimize(n_iter=1)[source]#

Run SciPy minimization with random restarts.

Each iteration draws a fresh starting point uniformly from the bounds and runs scipy.optimize.minimize once; the result with the lowest objective value across all restarts is returned.

Parameters:

n_iter (int) – Number of random restarts. Each restart is an independent SciPy minimization from a new random initial point; more restarts increase the chance of escaping poor local minima.

Returns:

The best result across restarts, with res.x unflattened back into the same structure as bounds (a tuple for sequence bounds, a dict for dict bounds). If every restart yields a non-finite objective (e.g. a diverging loss), the first such result is returned with its non-finite res.fun rather than None.

Return type:

scipy.optimize.OptimizeResult

Raises:

AssertionError – If n_iter is not a positive integer.