SlayerGrad#

class braintools.surrogate.SlayerGrad(alpha=1.0)#

Judge spiking state with the slayer surrogate gradient function.

The SLAYER (Spike LAYer Error Reassignment) gradient provides an exponential decay surrogate that enables error backpropagation in spiking neural networks. It uses a Laplace-like distribution for the gradient, offering a good balance between gradient magnitude near the threshold and computational efficiency. This surrogate was introduced in [1].

The forward function:

\[\begin{split}g(x) = \begin{cases} 1, & x \geq 0 \\ 0, & x < 0 \\ \end{cases}\end{split}\]

Backward gradient:

\[g'(x) = \exp(-\alpha \cdot |x|)\]

This creates an exponentially decaying gradient with:

  • Peak value of 1 at x=0

  • Exponential decay rate controlled by α

  • Symmetric profile around the threshold

>>> import jax
>>> import jax.numpy as jnp
>>> import brainstate
>>> import braintools.surrogate as surrogate
>>> import matplotlib.pyplot as plt
>>>
>>> xs = jnp.linspace(-4, 4, 1000)
>>> fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(10, 4))
>>>
>>> # Plot gradients for different alpha values
>>> for alpha in [0.5, 1.0, 2.0, 4.0]:
...     sg_fn = surrogate.SlayerGrad(alpha=alpha)
...     grads = jax.vmap(jax.grad(sg_fn))(xs)
...     _ = ax1.plot(xs, grads, label=rf'$\alpha={alpha}$')
>>>
>>> _ = ax1.set_xlabel('Input (x)')
>>> _ = ax1.set_ylabel('Gradient')
>>> _ = ax1.set_title('SLAYER Surrogate Gradients')
>>> _ = ax1.legend()
>>> ax1.grid(True, alpha=0.3)
>>>
>>> # Compare decay rates on semi-log plot
>>> xs_pos = jnp.linspace(0, 4, 500)
>>> for alpha in [0.5, 1.0, 2.0, 4.0]:
...     sg_fn = surrogate.SlayerGrad(alpha=alpha)
...     grads = jax.vmap(jax.grad(sg_fn))(xs_pos)
...     _ = ax2.semilogy(xs_pos, grads, label=rf'$\alpha={alpha}$')
>>>
>>> # Add theoretical exponential decay lines
>>> for alpha in [1.0, 2.0]:
...     theoretical = jnp.exp(-alpha * xs_pos)
...     _ = ax2.semilogy(xs_pos, theoretical, '--', alpha=0.5)
>>>
>>> _ = ax2.set_xlabel('Distance from threshold')
>>> _ = ax2.set_ylabel('Gradient (log scale)')
>>> _ = ax2.set_title('Exponential Decay Behavior')
>>> _ = ax2.legend()
>>> ax2.grid(True, alpha=0.3, which="both")
>>> plt.tight_layout()
>>> plt.show()

(Source code, png, hires.png, pdf)

../../_images/braintools-surrogate-SlayerGrad-1.png
Parameters:

alpha (float, optional) –

Parameter to control the decay rate of the gradient. Default is 1.0.

  • Larger α creates faster decay (sharper gradients)

  • Smaller α creates slower decay (wider gradients)

  • Decay length scale = 1/α

Examples

>>> import jax
>>> import braintools.surrogate as surrogate
>>>
>>> # Create SLAYER gradient surrogate
>>> sg_fn = surrogate.SlayerGrad(alpha=1.0)
>>>
>>> # Apply to input
>>> x = jax.numpy.array([-2., -1., 0., 1., 2.])
>>> spikes = sg_fn(x)
>>> print(spikes)
[0. 0. 1. 1. 1.]
>>>
>>> # Compute gradients showing exponential decay
>>> grad_fn = jax.grad(lambda x: sg_fn(x).sum())
>>> grads = grad_fn(x)
>>> print("Gradients:", ", ".join(f"{g:.4f}" for g in grads))
Gradients: 0.1353, 0.3679, 1.0000, 0.3679, 0.1353
>>> # Shows exp(-|x|) behavior

See also

slayer_grad

Functional version of SLAYER gradient.

GaussianGrad

Gaussian-based surrogate gradient.

InvSquareGrad

Power-law decay surrogate gradient.

References

surrogate_grad(x)[source]#

The gradient of the surrogate function, used during backpropagation.