Softmax

Contents

Softmax#

class brainstate.nn.Softmax(dim=None)#

Applies the Softmax function to an n-dimensional input Tensor.

Rescales the input so that the elements of the n-dimensional output Tensor lie in the range [0,1] and sum to 1.

Softmax is defined as:

\[\text{Softmax}(x_{i}) = \frac{\exp(x_i)}{\sum_j \exp(x_j)}\]

When the input Tensor is a sparse tensor then the unspecified values are treated as -inf.

Parameters:

  • dim (int | None) – A dimension along which Softmax will be computed (so every slice along dim will sum to 1).

  • Shape

  • -----

  • Input (-)

  • Output (-)

Returns:

A Tensor of the same dimension and shape as the input with values in the range [0, 1]

Return type:

Tensor

Notes

This module doesn’t work directly with NLLLoss, which expects the Log to be computed between the Softmax and itself. Use LogSoftmax instead (it’s faster and has better numerical properties).

Examples

>>> import brainstate.nn as nn
>>> import brainstate
>>> m = nn.Softmax(dim=1)
>>> x = brainstate.random.randn(2, 3)
>>> output = m(x)
Contents