LeakyReLU

Contents

LeakyReLU#

class brainstate.nn.LeakyReLU(negative_slope=0.01)#

Applies the element-wise function.

\[\text{LeakyReLU}(x) = \max(0, x) + \text{negative\_slope} * \min(0, x)\]

or

\[\begin{split}\text{LeakyReLU}(x) = \begin{cases} x, & \text{ if } x \geq 0 \\ \text{negative\_slope} \times x, & \text{ otherwise } \end{cases}\end{split}\]
Parameters:

  • negative_slope (float) – Controls the angle of the negative slope (which is used for negative input values). Default: 1e-2

  • Shape

  • -----

  • Input (-) – dimensions

  • Output (-)

Examples

>>> import brainstate.nn as nn
>>> import brainstate
>>> m = nn.LeakyReLU(0.1)
>>> x = brainstate.random.randn(2)
>>> output = m(x)
Contents