ReluT

class brainstate.nn.ReluT(lower_bound=0.0)

ReLU transform with lower bound: forward(x) = relu(x) + lower_bound

Constrains parameter values to be >= lower_bound. Note: inverse is not differentiable at x = lower_bound.

Parameters:

lower_bound (float) – Minimum value for the output (default: 0.0).

forward(x)

Apply the forward transformation.

Transforms input from the unconstrained domain to the constrained domain. This method implements the mathematical function \(y = f(x)\) where \(x\) is in the unconstrained space and \(y\) is in the target domain.

Parameters:

x (Array | ndarray | bool | number | bool | int | float | complex | Quantity) – Input array in the unconstrained domain.

Returns:

Transformed output in the constrained domain.

Return type:

Array

Notes

Implementations must ensure numerical stability and handle boundary conditions appropriately.

inverse(y)

Apply the inverse transformation.

Transforms input from the constrained domain back to the unconstrained domain. This method implements the mathematical function \(x = f^{-1}(y)\) where \(y\) is in the constrained space and \(x\) is in the unconstrained domain.

Parameters:

y (Array | ndarray | bool | number | bool | int | float | complex | Quantity) – Input array in the constrained domain.

Returns:

Transformed output in the unconstrained domain.

Return type:

Array

Notes

Implementations must ensure that inverse(forward(x)) = x for all valid x, and forward(inverse(y)) = y for all y in the target domain.