KaimingUniform

Contents

KaimingUniform#

class braintools.init.KaimingUniform(scale=None, mode='fan_in', nonlinearity='relu', negative_slope=0.01, unit=None)#

Kaiming/He uniform initialization.

Samples from a uniform distribution with bounds computed to maintain variance across layers. Recommended for ReLU and leaky ReLU activations.

Reference: He et al., “Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification”, ICCV 2015.

Parameters:

  • scale (Array | ndarray | bool | number | bool | int | float | complex | Quantity) – Variance multiplier (variance = scale / fan). When None (the default), it is derived from nonlinearity as the He variance multiplier gain ** 2 (2.0 for ReLU, 2 / (1 + negative_slope ** 2) for leaky ReLU). Pass an explicit value to override.

  • mode (Literal['fan_in', 'fan_out', 'fan_avg']) – Mode for computing scale factor (default: ‘fan_in’).

  • nonlinearity (Literal['relu', 'leaky_relu']) – Type of nonlinearity (default: ‘relu’). For leaky_relu, the scale is computed based on the negative slope.

  • negative_slope (float) – Negative slope for leaky_relu (default: 0.01).

Examples

>>> import numpy as np
>>> from braintools.init import KaimingUniform
>>>
>>> init = KaimingUniform()
>>> rng = np.random.default_rng(0)
>>> weights = init((100, 50), rng=rng)
Contents