StudentTReg#
- class brainstate.nn.StudentTReg(weight=1.0, df=3.0, scale=1.0, fit_hyper=False)#
Student’s t-distribution prior regularization.
Implements regularization based on the negative log-likelihood of a Student’s t-distribution, which has heavier tails than Gaussian:
\[L = \lambda \left[ \sum_i \frac{\nu + 1}{2} \log\left(1 + \frac{(x_i / s)^2}{\nu}\right) + N \left( \log s + \log\Gamma(\tfrac{\nu}{2}) + \tfrac{1}{2}\log(\nu\pi) - \log\Gamma(\tfrac{\nu+1}{2}) \right) \right]\]where \(\nu\) is the degrees of freedom, \(s\) is the scale, and \(N\) is the number of elements. The \(\log s\) and \(\Gamma\)-based terms form the scale/df log-normalizer, which keeps the loss bounded below when
scale/dfare trained (fit_hyper=True).- Parameters:
Examples
>>> import jax.numpy as jnp >>> from brainstate.nn import StudentTReg >>> reg = StudentTReg(weight=1.0, df=3.0, scale=1.0) >>> value = jnp.array([0.5, 2.0, -1.0]) >>> loss = reg.loss(value)
Notes
Student’s t prior is more robust to outliers than Gaussian. As df -> infinity, it approaches a Gaussian distribution. df=1 gives the Cauchy distribution.