GammaReg

class brainstate.nn.GammaReg(weight=1.0, alpha=2.0, beta=1.0, fit_hyper=False)

Gamma prior regularization (for positive parameters).

Implements regularization based on the negative log-likelihood of a Gamma distribution:

\[L = \lambda \left[ \sum_i \left(-(\alpha - 1) \log x_i + \beta x_i\right) + N \left( -\alpha \log\beta + \log\Gamma(\alpha) \right) \right]\]

where \(N\) is the number of elements. The \(-\alpha \log\beta + \log\Gamma(\alpha)\) term is the log-normalizer that keeps the loss bounded below when alpha/beta are trained.

Parameters:

• weight (float) – Regularization weight (lambda). Default is 1.0.

• alpha (float) – Shape parameter. Default is 2.0.

• beta (float) – Rate parameter. Default is 1.0.

• fit_hyper (bool) – Whether to optimize hyperparameters. Default is False.

Examples

>>> import jax.numpy as jnp
>>> from brainstate.nn import GammaReg
>>> reg = GammaReg(weight=1.0, alpha=2.0, beta=1.0)
>>> value = jnp.array([0.5, 1.0, 2.0])  # positive values
>>> loss = reg.loss(value)

Notes

Gamma prior is flexible for positive parameters. alpha=1 gives exponential distribution. The mode is (alpha-1)/beta for alpha >= 1.

loss(value)

Calculate Gamma regularization loss.

Parameters:

value (Array | ndarray | bool | number | bool | int | float | complex | Quantity) – Parameter values (should be positive).

Returns:

Gamma negative log-likelihood loss.

Return type:

Array | ndarray | bool | number | bool | int | float | complex | Quantity

reset_value()

Return the mode of Gamma ((alpha-1)/beta for alpha >= 1).

Returns:

Mode value.

Return type:

Array | ndarray | bool | number | bool | int | float | complex | Quantity

sample_init(shape)

Sample from Gamma distribution.

Parameters:

shape (int | Sequence[int] | integer | Sequence[integer]) – Shape of the sample.

Returns:

Sample from Gamma(alpha, beta).

Return type:

Array | ndarray | bool | number | bool | int | float | complex | Quantity