prelu

Contents

prelu#

class brainstate.nn.prelu(x, a=0.25)[source]#

Parametric Rectified Linear Unit activation function.

Applies the element-wise function:

\[\text{PReLU}(x) = \max(0,x) + a * \min(0,x)\]

or equivalently:

\[\begin{split}\text{PReLU}(x) = \begin{cases} x, & \text{ if } x \geq 0 \\ ax, & \text{ otherwise } \end{cases}\end{split}\]
Parameters:

  • x (ArrayLike) – Input array.

  • a (float or ArrayLike, optional) – The negative slope coefficient. Can be a learnable parameter. Default is 0.25.

Returns:

Output array with the same shape as the input.

Return type:

jax.Array or Quantity

Notes

When used in neural network layers, \(a\) can be a learnable parameter that is optimized during training.

Contents