KaimingNormal

Contents

KaimingNormal#

class braintools.init.KaimingNormal(scale=None, mode='fan_in', nonlinearity='relu', negative_slope=0.01, unit=None)#

Kaiming/He normal initialization.

Samples from a normal distribution with standard deviation computed to maintain variance across layers. Recommended for ReLU and leaky ReLU activations.

Reference: He et al., “Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification”, ICCV 2015.

Parameters:

  • mode (Literal['fan_in', 'fan_out', 'fan_avg']) – Mode for computing scale factor (default: ‘fan_in’).

  • nonlinearity (Literal['relu', 'leaky_relu']) – Type of nonlinearity (default: ‘relu’). For leaky_relu, the scale is computed based on the negative slope.

  • negative_slope (float) – Negative slope for leaky_relu (default: 0.01).

Examples

>>> import numpy as np
>>> from braintools.init import KaimingNormal
>>>
>>> init = KaimingNormal()
>>> rng = np.random.default_rng(0)
>>> weights = init((100, 50), rng=rng)
Contents