glu

class brainstate.nn.glu(x, axis=-1)

Gated Linear Unit activation function.

Computes the function:

\[\mathrm{glu}(x) = x\left[\ldots, 0:\frac{n}{2}, \ldots\right] \cdot \mathrm{sigmoid} \left( x\left[\ldots, \frac{n}{2}:n, \ldots\right] \right)\]

where the array is split into two along axis. The size of the axis dimension must be divisible by two.

Parameters:

• x (Array | ndarray | bool | number | bool | int | float | complex | Quantity) – Input array. The dimension specified by axis must be divisible by 2.

• axis (int) – The axis along which the split should be computed. Default is -1.

Returns:

An array with the same shape as input except the axis dimension is halved.

Return type:

Array | Quantity

See also

sigmoid

The sigmoid activation function.