leaky_relu

Contents

leaky_relu#

class brainunit.math.leaky_relu(x, negative_slope=0.01)#

Leaky rectified linear unit activation function.

Computes the element-wise function:

\[\begin{split}\mathrm{leaky\_relu}(x) = \begin{cases} x, & x \ge 0\\ \alpha x, & x < 0 \end{cases}\end{split}\]

where \(\alpha\) = negative_slope.

Parameters:

  • x (Quantity | Array | ndarray | bool | number | bool | int | float | complex) – Input array. If a Quantity with physical units is provided, the units are preserved in the output.

  • negative_slope (Array | ndarray | bool | number | bool | int | float | complex) – Slope for negative input values. Default is 0.01.

Returns:

out – An array with the same shape as x.

Return type:

Quantity | Array

Examples

>>> import jax.numpy as jnp
>>> import saiunit.math as sumath
>>> sumath.leaky_relu(jnp.array([-2., -1., 0., 1., 2.]))
Array([-0.02, -0.01,  0.  ,  1.  ,  2.  ], dtype=float32)
>>> sumath.leaky_relu(jnp.array([-1., 1.]), negative_slope=0.1)
Array([-0.1,  1. ], dtype=float32)
Contents