leaky_relu

class brainstate.nn.leaky_relu(x, negative_slope=0.01)

Leaky Rectified Linear Unit activation function.

Computes the element-wise function:

\[\begin{split}\mathrm{leaky\_relu}(x) = \begin{cases} x, & x \ge 0\\ \alpha x, & x < 0 \end{cases}\end{split}\]

where \(\alpha\) = negative_slope.

Parameters:

• x (Array | ndarray | bool | number | bool | int | float | complex | Quantity) – Input array.

• negative_slope (Array | ndarray | bool | number | bool | int | float | complex | Quantity) – Array or scalar specifying the negative slope. Default is 0.01.

Returns:

An array with the same shape as the input.

Return type:

Array | Quantity

See also

relu

Standard ReLU activation function.

prelu

Parametric ReLU with learnable slope.