Custom Surrogate Gradients#
A new surrogate takes a few lines: subclass braintools.surrogate.Surrogate and implement surrogate_grad. This notebook covers:
writing and checking a custom surrogate;
what the base class provides (jit, vmap, forward- and reverse-mode, higher derivatives) compared with a plain
jax.custom_vjp;surrogate parameters under
jax.jit, and how to change them during training;an asymmetric surrogate, and why the support decides whether silent neurons can learn.
1. Writing a Surrogate#
import jax
import jax.numpy as jnp
import matplotlib.pyplot as plt
import numpy as np
import braintools.surrogate as surrogate
jax.config.update("jax_enable_x64", True)
As an example, a raised-cosine surrogate of half-width \(w\): smooth, with finite support and mass 1,
surrogate_grad is required; surrogate_fun is optional and only used for plotting and analysis. Store parameters as attributes.
class RaisedCosine(surrogate.Surrogate):
def __init__(self, width=1.0):
super().__init__()
self.width = width
def surrogate_fun(self, x):
z = jnp.clip(x / self.width, -1.0, 1.0)
return 0.5 * (1.0 + z + jnp.sin(jnp.pi * z) / jnp.pi)
def surrogate_grad(self, x):
inside = jnp.abs(x) < self.width
return jnp.where(inside, (1.0 + jnp.cos(jnp.pi * x / self.width)) / (2.0 * self.width), 0.0)
def __repr__(self):
return f"{self.__class__.__name__}(width={self.width})"
sg = RaisedCosine(width=0.5)
xs = jnp.linspace(-1.5, 1.5, 601)
print(sg)
print("forward:", sg(jnp.array([-0.2, 0.0, 0.2])))
RaisedCosine(width=0.5)
forward: [0. 1. 1.]
Three checks catch most mistakes: jax.grad of the spike equals surrogate_grad; surrogate_grad is the derivative of surrogate_fun; and the mass \(\int\sigma'\) is what you intended.
g_spike = jax.vmap(jax.grad(sg))(xs)
g_fun = jax.vmap(jax.grad(sg.surrogate_fun))(xs)
print("jax.grad(spike) == surrogate_grad :", bool(jnp.allclose(g_spike, sg.surrogate_grad(xs))))
print("d surrogate_fun / dx == surrogate_grad:", bool(jnp.allclose(g_fun, sg.surrogate_grad(xs), atol=1e-6)))
print("mass:", float(jnp.trapezoid(sg.surrogate_grad(xs), xs)))
fig, ax = plt.subplots(1, 2, figsize=(11, 3.5))
ax[0].plot(xs, sg.surrogate_fun(xs), label="surrogate_fun"); ax[0].plot(xs, sg(xs), "k--", lw=1, label="forward H(x)")
ax[1].plot(xs, sg.surrogate_grad(xs), label="RaisedCosine(0.5)")
ax[1].plot(xs, surrogate.Sigmoid(alpha=4.0).surrogate_grad(xs), label="Sigmoid(4)")
for a in ax: a.legend(); a.set_xlabel("x")
plt.tight_layout(); plt.show()
jax.grad(spike) == surrogate_grad : True
d surrogate_fun / dx == surrogate_grad: True
mass: 1.0000000000000002
2. What the Base Class Provides#
Surrogate.__call__ binds a JAX primitive whose forward pass is the step and whose derivative rule multiplies the input tangent by surrogate_grad(x). It therefore works with every JAX transformation, including forward mode and higher derivatives. A plain jax.custom_vjp function with the same backward rule supports reverse mode (including repeated reverse mode), but not forward mode:
@jax.custom_vjp
def spike_vjp(x):
return (x >= 0).astype(x.dtype)
spike_vjp.defvjp(lambda x: (spike_vjp(x), x), lambda x, ct: (ct * sg.surrogate_grad(x),))
x0 = jnp.array([-0.2, 0.1, 0.3])
checks = {
"reverse mode (jax.grad)": lambda f: jax.vmap(jax.grad(f))(x0),
"forward mode (jax.jacfwd)": lambda f: jnp.diag(jax.jacfwd(lambda x: f(x))(x0)),
"second derivative": lambda f: jax.vmap(jax.grad(jax.grad(f)))(x0),
"jit + vmap": lambda f: jax.jit(jax.vmap(jax.grad(f)))(x0),
}
for label, run in checks.items():
row = []
for f in [sg, spike_vjp]:
try:
row.append(np.round(np.asarray(run(f)), 4))
except Exception as e:
row.append(type(e).__name__)
print(f"{label:26s} Surrogate: {row[0]} custom_vjp: {row[1]}")
reverse mode (jax.grad) Surrogate: [1.309 1.809 0.691] custom_vjp: [1.309 1.809 0.691]
forward mode (jax.jacfwd) Surrogate: [1.309 1.809 0.691] custom_vjp: TypeError
second derivative Surrogate: [ 5.9757 -3.6932 -5.9757] custom_vjp: [ 5.9757 -3.6932 -5.9757]
jit + vmap Surrogate: [1.309 1.809 0.691] custom_vjp: [1.309 1.809 0.691]
Forward mode matters for forward-mode training methods (for example braintools.optim.SOFO, which uses Jacobian-vector products) and for jax.jacfwd/jax.hessian. The second derivative is the derivative of surrogate_grad, which appears when differentiating through a training step (third notebook).
Surrogates are also hashable and compared by value (same class, equal parameters), so passing a new but equal surrogate as a static argument to jax.jit reuses the compiled function.
3. Parameters Under jax.jit#
Attributes such as width are Python values. A jitted function that uses a surrogate object captures their values when it is traced; changing the attribute afterwards has no effect on the compiled function, and no error is raised:
sg = RaisedCosine(width=0.5)
grad_jit = jax.jit(jax.grad(lambda x: sg(x)))
print("width 0.5: ", float(grad_jit(0.1)))
sg.width = 1.0
print("after sg.width = 1.0 (jitted): ", float(grad_jit(0.1)), " <- still the old width")
print("after sg.width = 1.0 (no jit): ", float(jax.grad(lambda x: sg(x))(0.1)))
width 0.5: 1.8090169943749475
after sg.width = 1.0 (jitted): 1.8090169943749475 <- still the old width
after sg.width = 1.0 (no jit): 0.9755282581475768
To change a parameter during training, for example to anneal the width, pass it as an argument and construct the surrogate inside the jitted function (or use the functional forms such as surrogate.sigmoid(x, alpha)). The parameter is then a traced value: new values reuse the same compiled code.
@jax.jit
def spike_grad(x, width):
return jax.grad(lambda x: RaisedCosine(width)(x))(x)
for width in [2.0, 1.0, 0.5, 0.25]:
print(f"width {width:4}: d spike/dx at x = 0.1 is {float(spike_grad(0.1, width)):.4f}")
print("compilations:", spike_grad._cache_size())
width 2.0: d spike/dx at x = 0.1 is 0.4969
width 1.0: d spike/dx at x = 0.1 is 0.9755
width 0.5: d spike/dx at x = 0.1 is 1.8090
width 0.25: d spike/dx at x = 0.1 is 2.6180
compilations: 1
4. An Asymmetric Surrogate for Silent Neurons#
With a finite-support surrogate, a neuron whose potential stays far below threshold passes no gradient at all, so the weights feeding it never change: it is dead. Here a LIF neuron receives Poisson input through 20 synapses whose weights are initially too weak to make it fire, and we train the weights to produce 10 output spikes in 500 ms. We compare a finite-support surrogate, Sigmoid, and a custom surrogate with a long left tail that stays narrow above threshold.
class LongLeftTail(surrogate.Surrogate):
# Sigmoid-shaped above threshold, slowly decaying 1 / (1 + |x| / s)^2 tail below it
def __init__(self, alpha=4.0, s=1.0):
super().__init__()
self.alpha, self.s = alpha, s
def surrogate_grad(self, x):
sig = jax.nn.sigmoid(self.alpha * x)
right = self.alpha * sig * (1.0 - sig)
left = (self.alpha / 4.0) / (1.0 + jnp.abs(x) / self.s) ** 2
return jnp.where(x >= 0, right, left)
key = jax.random.PRNGKey(0)
n_syn, dt, tau, n_steps = 20, 1.0, 20.0, 500
inputs = (jax.random.uniform(key, (n_steps, n_syn)) < 0.02).astype(float) # 20 Hz Poisson inputs
def output_spikes(w, sg):
def step(v, x_t):
v = v + dt / tau * (-v) + x_t @ w
s = sg(v - 1.0)
return v - s, (s, v)
_, (spikes, vs) = jax.lax.scan(step, 0.0, inputs)
return spikes.sum(), vs
w0 = jnp.full(n_syn, 0.01)
print(f"initial: {float(output_spikes(w0, surrogate.Sigmoid())[0]):.0f} spikes, max potential "
f"{float(output_spikes(w0, surrogate.Sigmoid())[1].max()):.2f} (threshold 1)")
fig, ax = plt.subplots(1, 2, figsize=(12, 3.8))
xs = jnp.linspace(-3, 1.5, 451)
candidates = {"PiecewiseQuadratic(2)": surrogate.PiecewiseQuadratic(alpha=2.0), "Sigmoid(4)": surrogate.Sigmoid(4.0),
"LongLeftTail": LongLeftTail()}
for name, sg_ in candidates.items():
ax[0].plot(xs, sg_.surrogate_grad(xs), label=name)
loss = jax.jit(jax.value_and_grad(lambda w: (output_spikes(w, sg_)[0] - 10.0) ** 2))
w, history = w0, []
for it in range(60):
l, g = loss(w)
history.append(float(output_spikes(w, sg_)[0]))
w = w - 1e-4 * g
ax[1].plot(history, label=name)
print(f"{name:20s} initial gradient norm {float(jnp.linalg.norm(loss(w0)[1])):9.3e} spikes after 60 steps: {history[-1]:.0f}")
ax[0].set_xlabel("x = V - V_th"); ax[0].set_title("surrogate_grad"); ax[0].legend()
ax[1].axhline(10, color="k", ls="--", lw=1); ax[1].set_xlabel("gradient step"); ax[1].set_ylabel("output spikes"); ax[1].legend()
plt.tight_layout(); plt.show()
initial: 0 spikes, max potential 0.14 (threshold 1)
PiecewiseQuadratic(2) initial gradient norm 0.000e+00 spikes after 60 steps: 0
Sigmoid(4) initial gradient norm 6.517e+02 spikes after 60 steps: 10
LongLeftTail initial gradient norm 8.488e+02 spikes after 60 steps: 10
With PiecewiseQuadratic(alpha=2), whose gradient vanishes for \(|x| > 0.5\), the potential never comes within reach of the support (its maximum is 0.14, so \(x \le -0.86\)): the gradient is exactly zero and training never starts. Sigmoid(4) has only an exponentially thin tail at \(x \approx -0.9\), but summed over 500 time steps it is enough, and the long-tailed surrogate trains only slightly faster.
The lesson is about the support, not about this particular tail: a neuron can only learn if the surrogate is non-zero where its potential actually is. With finite-support surrogates, initialize weights (or thresholds) such that neurons come close to threshold, or choose a surrogate with tails. Note also that the scale of the threshold matters: x is \(V - V_{th}\) in whatever units the model uses.
Summary#
Subclass
Surrogateand implementsurrogate_grad(x); addsurrogate_fun(x)if you want to plot or analyse the smooth function. Check thatjax.gradof the spike equalssurrogate_grad, thatsurrogate_gradis the derivative ofsurrogate_fun, and the mass.The base class gives reverse mode, forward mode, higher derivatives,
jitandvmap, and value-based equality; a plainjax.custom_vjplacks forward mode.Surrogate attributes are frozen into compiled code. To vary a parameter during training, pass it as an argument and build the surrogate inside the jitted function.
The support of the surrogate decides whether silent neurons can learn: where the surrogate is exactly zero, they receive no gradient.