relu

Contents

relu#

class brainstate.nn.relu(x)[source]#

Rectified Linear Unit activation function.

Computes the element-wise function:

\[\mathrm{relu}(x) = \max(x, 0)\]

Under differentiation, we take:

\[\nabla \mathrm{relu}(0) = 0\]
Parameters:

x (Array | ndarray | bool | number | bool | int | float | complex | Quantity) – Input array.

Returns:

An array with the same shape as the input.

Return type:

Array | Quantity

Examples

>>> import jax.numpy as jnp
>>> import brainstate
>>> brainstate.nn.relu(jnp.array([-2., -1., -0.5, 0, 0.5, 1., 2.]))
Array([0. , 0. , 0. , 0. , 0.5, 1. , 2. ], dtype=float32)

See also

relu6

ReLU6 activation function.

leaky_relu

Leaky ReLU activation function.

References

Contents