ReLU

Contents

ReLU#

class brainstate.nn.ReLU(name=None)#

Applies the rectified linear unit function element-wise.

The ReLU function is defined as:

\[\text{ReLU}(x) = (x)^+ = \max(0, x)\]

Shape#

  • Input: \((*)\), where \(*\) means any number of dimensions.

  • Output: \((*)\), same shape as the input.

Examples

>>> import brainstate.nn as nn
>>> import brainstate
>>> m = nn.ReLU()
>>> x = brainstate.random.randn(2)
>>> output = m(x)

An implementation of CReLU - https://arxiv.org/abs/1603.05201

>>> import brainstate.nn as nn
>>> import brainstate
>>> import jax.numpy as jnp
>>> m = nn.ReLU()
>>> x = brainstate.random.randn(2).unsqueeze(0)
>>> output = jnp.concat((m(x), m(-x)))
Contents