SoftplusT

class brainstate.nn.SoftplusT(lower)

Softplus transformation mapping unbounded values to positive semi-infinite interval.

This transformation uses the softplus function to map any real value to the interval [lower, ∞). It provides a smooth, differentiable alternative to ReLU activation and is commonly used to constrain parameters to be positive, such as variance parameters or rate constants.

The transformation is defined by:

\[\text{forward}(x) = \log(1 + e^x) + \text{lower}\]

The inverse transformation is:

\[\text{inverse}(y) = \log(e^{y - \text{lower}} - 1)\]

Parameters:

lower (Array | ndarray | bool | number | bool | int | float | complex | Quantity) – Lower bound of the target interval.

lower

Lower bound of the interval.

Type:

array_like

unit

Physical unit of the lower bound.

Type:

brainunit.Unit

Notes

The softplus function is the smooth approximation to the ReLU function: \(\lim_{\beta \to \infty} \frac{1}{\beta} \log(1 + e^{\beta x}) = \max(0, x)\)

For large positive x, softplus(x) ≈ x, and for large negative x, softplus(x) ≈ 0. The function is strictly positive and has a well-defined inverse.

Examples

>>> # Map to positive reals [0, ∞)
>>> transform = SoftplusT(0.0)
>>> x = jnp.array([-5.0, 0.0, 5.0])
>>> y = transform.forward(x)
>>> # y ≈ [0.007, 0.693, 5.007]

>>> # Map to interval [2, ∞) for positive-definite parameters
>>> transform = SoftplusT(2.0)
>>> x = jnp.array([0.0])
>>> y = transform.forward(x)
>>> # y ≈ [2.693]

forward(x)

Transform unbounded input to positive semi-infinite interval.

Parameters:

x (Array | ndarray | bool | number | bool | int | float | complex | Quantity) – Input values in unbounded domain (-∞, ∞).

Returns:

Transformed values in interval [lower, ∞).

Return type:

Array

Notes

Uses log1p for numerical stability: log1p(exp(x)) = log(1 + exp(x)). For large x, this avoids overflow in the exponential.

inverse(y)

Transform positive semi-infinite input back to unbounded domain.

Parameters:

y (Array | ndarray | bool | number | bool | int | float | complex | Quantity) – Input values in interval [lower, ∞).

Returns:

Transformed values in unbounded domain (-∞, ∞).

Return type:

Array

Notes

Input must be strictly greater than lower bound to avoid numerical issues. Uses numerically stable exponential for large (y - lower) values.

log_abs_det_jacobian(x, y)

Compute log absolute determinant of the Jacobian.

For softplus: d/dx[log(1 + exp(x))] = sigmoid(x) log|det J| = sum(log(sigmoid(x))) = sum(x - softplus(x))

Return type:

Array