huber_loss

Contents

huber_loss#

class braintools.metric.huber_loss(predictions, targets=None, delta=1.0, axis=None, reduction='none')#

Compute Huber loss combining L1 and L2 properties for robust regression.

The Huber loss provides a compromise between L1 and L2 losses, being quadratic for small errors (like L2) and linear for large errors (like L1). This makes it robust to outliers while maintaining smooth gradients near zero, combining the best properties of both loss functions.

The Huber loss is defined as:

\[\begin{split}\ell_\delta(a) = \begin{cases} \frac{1}{2} a^2 & \text{if } |a| \leq \delta \\ \delta |a| - \frac{1}{2} \delta^2 & \text{if } |a| > \delta \end{cases}\end{split}\]

where \(a = y - \hat{y}\) is the residual and \(\delta\) is the threshold.

Parameters:

  • predictions (Array | ndarray | bool | number | bool | int | float | complex | Quantity) – Predicted values with arbitrary shape. Must be floating-point type.

  • targets (Array | ndarray | bool | number | bool | int | float | complex | Quantity | None) – Ground truth target values with shape broadcastable to predictions. If not provided, targets are assumed to be zeros.

  • delta (float | Array | ndarray | bool | number | bool | int | complex | Quantity) – Threshold parameter that controls the transition between quadratic and linear regions. Smaller values make the loss more L1-like (robust but less smooth), while larger values make it more L2-like (smooth but less robust). May be a brainunit.Quantity whose units match those of the errors (see Notes).

  • axis (int | tuple[int, ...] | None) – Axis or axes along which to reduce when reduction is 'mean' or 'sum'. If None, reduction (if any) is over all elements.

  • reduction (str) –

    Reduction operation to apply:

    • 'none': return element-wise losses with the same shape as predictions (backward-compatible default),

    • 'mean': return the mean of the losses,

    • 'sum': return the sum of the losses.

Returns:

Element-wise Huber losses with the same shape as predictions when reduction='none'; otherwise the reduced value.

Return type:

Array | ndarray | bool | number | bool | int | float | complex | Quantity

Notes

The Huber loss has several important properties:

  • Robustness: Linear growth for large errors reduces outlier sensitivity

  • Smoothness: Quadratic near zero ensures smooth gradients for optimization

  • Gradient clipping: Equivalent to clipping L2 gradients to [-delta, delta]

The choice of delta parameter affects the balance:

  • Small delta: More robust, approaches L1 loss

  • Large delta: Less robust, approaches L2 loss

  • delta = 1.0: Common default providing good balance

This loss is particularly effective for regression with outliers and in reinforcement learning for value function approximation.

Units: comparisons are performed with brainunit.math so that a brainunit.Quantity delta is handled correctly. When the errors carry units, delta must carry the same units as the errors (a dimensionless delta against unit-bearing errors raises a unit-mismatch error). For Quantity inputs the loss carries squared units in the quadratic region.

Examples

Basic Huber loss computation:

>>> import jax.numpy as jnp
>>> import braintools
>>> predictions = jnp.array([1.0, 2.0, 5.0])
>>> targets = jnp.array([1.1, 1.9, 3.0])  # Last prediction is outlier
>>> braintools.metric.huber_loss(predictions, targets)
Array([0.005, 0.005, 1.5  ], dtype=float32)

Reduce to a scalar mean:

>>> braintools.metric.huber_loss(predictions, targets, reduction='mean')
Array(0.50333333, dtype=float32)

See also

braintools.metric.absolute_error

Pure L1 loss

braintools.metric.squared_error

Pure L2 loss

braintools.metric.log_cosh

Smooth alternative to Huber loss

References

Contents