VarianceScaling

Contents

VarianceScaling#

class braintools.init.VarianceScaling(scale=1.0, mode='fan_in', distribution='normal', unit=None, in_axis=-2, out_axis=-1)#

Base class for variance scaling initializations.

Variance scaling methods compute an appropriate scale factor based on the number of input and/or output units, then sample from a distribution with that scale.

Parameters:
  • scale (Array | ndarray | bool | number | bool | int | float | complex | Quantity) – Scaling factor (positive float).

  • mode (Literal['fan_in', 'fan_out', 'fan_avg']) – Mode for computing the scale factor: - ‘fan_in’: scale by number of input units - ‘fan_out’: scale by number of output units - ‘fan_avg’: scale by average of input and output units

  • distribution (Literal['uniform', 'normal', 'truncated_normal']) – Distribution to sample from.

  • in_axis (int) – Axis of the weight shape holding the input units (default: -2).

  • out_axis (int) – Axis of the weight shape holding the output units (default: -1). Every other axis is a receptive-field axis.

Notes

Fans follow the JAX/Flax convention used by brainstate layers: a dense weight is (in, out) and a convolution kernel is (*spatial, in, out), so fan_in = shape[in_axis] * receptive_field and fan_out = shape[out_axis] * receptive_field with receptive_field = prod(shape) / (shape[in_axis] * shape[out_axis]). A 1-D shape (n,) uses fan_in = fan_out = n and a 0-D shape uses 1. For channels-first kernels (out, in, *spatial) pass in_axis=1, out_axis=0.