VarianceScaling#
- class braintools.init.VarianceScaling(scale=1.0, mode='fan_in', distribution='normal', unit=None, in_axis=-2, out_axis=-1)#
Base class for variance scaling initializations.
Variance scaling methods compute an appropriate scale factor based on the number of input and/or output units, then sample from a distribution with that scale.
- Parameters:
scale (
Array|ndarray|bool|number|bool|int|float|complex|Quantity) – Scaling factor (positive float).mode (
Literal['fan_in','fan_out','fan_avg']) – Mode for computing the scale factor: - ‘fan_in’: scale by number of input units - ‘fan_out’: scale by number of output units - ‘fan_avg’: scale by average of input and output unitsdistribution (
Literal['uniform','normal','truncated_normal']) – Distribution to sample from.in_axis (
int) – Axis of the weight shape holding the input units (default: -2).out_axis (
int) – Axis of the weight shape holding the output units (default: -1). Every other axis is a receptive-field axis.
Notes
Fans follow the JAX/Flax convention used by
brainstatelayers: a dense weight is(in, out)and a convolution kernel is(*spatial, in, out), sofan_in = shape[in_axis] * receptive_fieldandfan_out = shape[out_axis] * receptive_fieldwithreceptive_field = prod(shape) / (shape[in_axis] * shape[out_axis]). A 1-D shape(n,)usesfan_in = fan_out = nand a 0-D shape uses 1. For channels-first kernels(out, in, *spatial)passin_axis=1, out_axis=0.