XavierNormal

Contents

XavierNormal#

class braintools.init.XavierNormal(scale=1.0, unit=None, in_axis=-2, out_axis=-1)#

Xavier/Glorot normal initialization.

Samples from a normal distribution with standard deviation computed to maintain variance across layers. Recommended for tanh and sigmoid activations.

Reference: Glorot & Bengio, “Understanding the difficulty of training deep feedforward neural networks”, AISTATS 2010.

Parameters:
  • scale (Array | ndarray | bool | number | bool | int | float | complex | Quantity) – Scaling factor (default: 1.0).

  • in_axis (int) – Axes holding the input / output units (default: -2 / -1, i.e. the (in, out) / (*spatial, in, out) convention of brainstate).

  • out_axis (int) – Axes holding the input / output units (default: -2 / -1, i.e. the (in, out) / (*spatial, in, out) convention of brainstate).

Examples

>>> import numpy as np
>>> from braintools.init import XavierNormal
>>>
>>> init = XavierNormal()
>>> rng = np.random.default_rng(0)
>>> weights = init((100, 50), rng=rng)