SELU

Contents

SELU#

class brainstate.nn.SELU(name=None)#

Applied element-wise.

\[\text{SELU}(x) = \text{scale} * (\max(0,x) + \min(0, \alpha * (\exp(x) - 1)))\]

with \(\alpha = 1.6732632423543772848170429916717\) and \(\text{scale} = 1.0507009873554804934193349852946\).

More details can be found in the paper Self-Normalizing Neural Networks .

Shape#

  • Input: \((*)\), where \(*\) means any number of dimensions.

  • Output: \((*)\), same shape as the input.

References

Examples

>>> import brainstate.nn as nn
>>> import brainstate
>>> m = nn.SELU()
>>> x = brainstate.random.randn(2)
>>> output = m(x)
Contents