CauchyReg

class brainstate.nn.CauchyReg(weight=1.0, scale=1.0, fit_hyper=False)

Cauchy prior regularization.

Implements regularization based on the negative log-likelihood of a Cauchy distribution (Student’s t with df=1), which has very heavy tails:

\[L = \lambda \sum_i \log\left(1 + (x_i / s)^2\right)\]

Parameters:

• weight (float) – Regularization weight (lambda). Default is 1.0.

• scale (float) – Scale parameter. Default is 1.0.

• fit_hyper (bool) – Whether to optimize hyperparameters. Default is False.

Examples

>>> import jax.numpy as jnp
>>> from brainstate.nn import CauchyReg
>>> reg = CauchyReg(weight=1.0, scale=1.0)
>>> value = jnp.array([0.5, 5.0, -1.0])
>>> loss = reg.loss(value)

Notes

Cauchy prior allows for very large parameter values, making it extremely robust but also allowing outliers. It has no defined mean or variance.

loss(value)

Calculate Cauchy regularization loss.

Parameters:

value (Array | ndarray | bool | number | bool | int | float | complex | Quantity) – Parameter values.

Returns:

Cauchy negative log-likelihood loss.

Return type:

Array | ndarray | bool | number | bool | int | float | complex | Quantity

reset_value()

Return zero (the mode of symmetric Cauchy).

Returns:

Zero.

Return type:

Array | ndarray | bool | number | bool | int | float | complex | Quantity

sample_init(shape)

Sample from Cauchy distribution.

Parameters:

shape (int | Sequence[int] | integer | Sequence[integer]) – Shape of the sample.

Returns:

Sample from Cauchy distribution.

Return type:

Array | ndarray | bool | number | bool | int | float | complex | Quantity