sparse_plus

class brainstate.nn.sparse_plus(x)

Sparse plus activation function.

Computes the function:

\[\begin{split}\mathrm{sparse\_plus}(x) = \begin{cases} 0, & x \leq -1\\ \frac{1}{4}(x+1)^2, & -1 < x < 1 \\ x, & 1 \leq x \end{cases}\end{split}\]

This is the twin function of the softplus activation, ensuring a zero output for inputs less than -1 and a linear output for inputs greater than 1, while remaining smooth, convex, and monotonic between -1 and 1.

Parameters:

x (Array | ndarray | bool | number | bool | int | float | complex | Quantity) – Input array.

Returns:

An array with the same shape as the input.

Return type:

Array | Quantity

See also

sparse_sigmoid

Derivative of sparse_plus.

softplus

Standard softplus activation function.