rate_transformer_node

rate_transformer_node#

class brainpy.state.rate_transformer_node(in_size, linear_summation=True, g=1.0, input_nonlinearity=None, rate_initializer=Constant(value=0.0), name=None)#

NEST-compatible rate_transformer_node template model.

A stateless rate-based processing node that aggregates weighted incoming rate signals and applies a configurable input nonlinearity. Serves as an intermediary transformation stage in rate-based neural networks, mimicking NEST’s rate_transformer_node<TNonlinearities> template.

1. Mathematical Model

The model implements the transformation:

\[X_i(t) = \phi\!\left(\sum_j w_{ij}\,\psi\!\left(X_j(t-d_{ij})\right)\right)\]

where:

\(X_i(t)\) is the output rate of node \(i\) at time \(t\)
\(X_j(t-d_{ij})\) are incoming rates from presynaptic nodes \(j\) with delay \(d_{ij}\)
\(w_{ij}\) are connection weights
\(\phi\) is the input nonlinearity (applied to summed input)
\(\psi\) is the output nonlinearity (applied per-event before summation)

The model has no intrinsic dynamics: no differential equations, no noise, no membrane time constant. It acts purely as a feedforward transformation stage.

2. Nonlinearity Application Modes

The linear_summation parameter controls where the nonlinearity is applied, matching NEST’s event handler semantics:

Mode A: ``linear_summation=True`` (default, recommended)

Event handlers store weighted rates without applying nonlinearity
Nonlinearity \(\phi\) is applied once to the summed input during update()
Efficient when many inputs converge to one node
Math: \(X_i = \phi\!\left(\sum_j w_{ij} X_j(t-d_{ij})\right)\)

Mode B: ``linear_summation=False``

Nonlinearity \(\psi\) is applied per-event before summation
Event handlers pre-transform each incoming rate
Useful for models where nonlinearity operates on individual inputs
Math: \(X_i = \sum_j w_{ij}\,\psi\!\left(X_j(t-d_{ij})\right)\)

3. Default Nonlinearity

If no custom input_nonlinearity is provided, the model uses a simple gain function:

\[\phi(h) = g \cdot h\]

where \(g\) is the g parameter (default 1.0).

4. Update Algorithm

dftype = brainstate.environ.dftype() ditype = brainstate.environ.ditype() The update sequence (matching NEST’s rate_transformer_node_impl.h) is:

Store delayed output: Copy current rate → delayed_rate for outgoing delayed connections
Drain delayed queue: Retrieve events scheduled for current timestep
Process delayed events: Handle delayed_rate_events with explicit delay specifications
Process instant events: Handle instant_rate_events (zero-delay)
Sum contributions: Aggregate all weighted inputs into a single value
Apply nonlinearity (if linear_summation=True): Transform summed input
Update state: Store new rate and instant_rate
Increment step counter: Advance internal timestep

5. Event Handling

The model accepts two types of runtime events in update():

Instant events (``instant_rate_events``):

Applied in the current timestep (zero delay)
Must not specify non-zero delay_steps (raises ValueError)
Typical use: direct feedforward connections

Delayed events (``delayed_rate_events``):

Stored in internal queue and applied after delay_steps timesteps
Default delay is 1 step if not specified
Negative delays raise ValueError

Event format (flexible tuple/dict):

2-tuple: (rate, weight) → uses default delay
3-tuple: (rate, weight, delay_steps)
4-tuple: (rate, weight, delay_steps, multiplicity)
dict: {'rate': ..., 'weight': ..., 'delay_steps': ..., 'multiplicity': ...}
Scalar: interpreted as rate with weight=1.0, delay=default, multiplicity=1.0

6. Latency Considerations

As in NEST, inserting a transformer node introduces one simulation step of latency compared to direct instantaneous connections. This is because:

Output instant_rate is updated at the end of the current timestep
Downstream nodes read this value in the next timestep
Even “instantaneous” connections have this inherent discrete-time delay

7. Computational Complexity

Time complexity: \(O(E)\) per timestep, where \(E\) is the number of events
Space complexity: \(O(D \times N)\) for delayed queue, where \(D\) is max delay and \(N\) is population size
Nonlinearity cost: \(O(N)\) per call (applied once if linear_summation=True, or \(E\) times otherwise)

Parameters:

in_size (int, tuple of int, or Size) – Shape of the node population. Can be a scalar (1D population), tuple (multi-dimensional), or brainstate.Size object. Determines the shape of rate state variable.
linear_summation (bool, optional) –
Controls where the input nonlinearity is applied:
- True (default): Apply nonlinearity once to summed input (efficient, recommended)
- False: Apply nonlinearity per-event before summation (mathematically different)
See “Nonlinearity Application Modes” above for detailed semantics.
g (float, array_like, optional) – Gain parameter for the default linear nonlinearity \(\phi(h) = g \cdot h\). Can be a scalar (shared across population) or array with shape matching in_size. Ignored if custom input_nonlinearity is provided. Default: 1.0 (identity transform).
input_nonlinearity (callable, optional) –
Custom input nonlinearity function replacing the default \(g \cdot h\). Must accept:
- Signature 1: f(h) where h is ndarray of summed inputs (shape in_size)
- Signature 2: f(model, h) where model is self (for accessing parameters)
The function is automatically inspected to determine which signature to use. If None (default), uses g * h. Return value must broadcast to in_size.
rate_initializer (callable, optional) –
Initializer for the rate state variable. Must be a callable accepting (shape, batch_size) and returning an array. Common choices:
- braintools.init.Constant(0.0) (default): Initialize to zero
- braintools.init.Normal(0.0, 0.1): Gaussian initialization
- braintools.init.Uniform(0.0, 1.0): Uniform random
Default: Constant(0.0) (all rates start at zero).
name (str, optional) – Unique identifier for this module instance. Used for logging, debugging, and visualization. If None, an auto-generated name is assigned. Default: None.

Parameter Mapping

This table maps brainpy.state parameters to NEST’s template instantiation:

brainpy.state parameter	NEST equivalent	Notes
`in_size`	(implicit in node creation)	Population size
`linear_summation=True`	`linear_summation=true`	Default mode in NEST
`linear_summation=False`	`linear_summation=false`	Per-event nonlinearity
`g`	(template parameter)	Gain in default nonlinearity
`input_nonlinearity`	`TNonlinearities::input`	Custom \(\phi\) or \(\psi\)
`rate_initializer`	(no direct equiv)	NEST defaults to 0.0

State Variables

ratendarray, shape (in_size,): Primary output rate of the node. Updated at the end of each timestep after applying nonlinearity. This is the value read by downstream connections in the next timestep. Type: ShortTermState (persists between timesteps but not saved in checkpoints).
instant_ratendarray, shape (in_size,): Instantaneous output rate, identical to rate after update. Provided for clarity in models that distinguish between instant and delayed outputs. Equals rate at the end of each timestep.
delayed_ratendarray, shape (in_size,): Previous timestep’s output rate, used for delayed outgoing connections. Equals rate from the previous timestep. Allows modeling connection delays explicitly.
_step_countint64 scalar: Internal timestep counter used for delayed event scheduling. Increments by 1 each update. Not intended for external access.

Receptor Types

The model exposes a single receptor type:

'RATE': 0 — accepts rate-valued inputs (both instant and delayed)

Use this in connection specifications to route rate signals to the transformer node.

Recordables

The following state variables can be monitored during simulation:

'rate' — primary output rate (most commonly recorded)

Notes

Comparison with NEST:

Fully replicates NEST’s rate_transformer_node_impl.h event handling logic
Supports all NEST template instantiations via custom input_nonlinearity
Uses NumPy-based event queue instead of NEST’s C++ ring buffer (functionally identical)
Batch processing is supported via init_state(batch_size=...)

When to use this model:

Layered rate networks: Inserting nonlinear transformations between rate-neuron populations
Gain modulation: Implementing attention or gating via spatially varying g
Modular architectures: Separating rate dynamics (in rate neurons) from transformations (in transformer nodes)

Failure modes:

Passing non-zero delay_steps in instant_rate_events raises ValueError
Negative delay_steps in delayed_rate_events raises ValueError
Custom input_nonlinearity returning wrong shape causes broadcasting errors

Performance tips:

Use linear_summation=True (default) for better efficiency when many inputs converge
Avoid unnecessary delayed events if connections are instantaneous
For very long delays, consider pruning old queue entries manually if memory is constrained

References

Examples

Example 1: Basic usage with default linear gain

>>> import brainpy_state as bst
>>> import saiunit as u
>>> import brainstate
>>> # Create a transformer node with 10 units
>>> transformer = bst.rate_transformer_node(in_size=10, g=2.0)
>>> # Initialize state
>>> transformer.init_state()
>>> # Send instant rate events (rate=0.5, weight=1.0)
>>> with brainstate.environ.context(dt=0.1 * u.ms):
...     output = transformer.update(instant_rate_events=(0.5, 1.0))
>>> print(output)
[1.0, 1.0, ..., 1.0]  # g * rate * weight = 2.0 * 0.5 * 1.0

Example 2: Custom sigmoid nonlinearity

>>> import numpy as np
>>> import brainpy_state as bst
>>> # Define sigmoid activation
>>> def sigmoid(h):
...     return 1.0 / (1.0 + np.exp(-h))
>>> transformer = bst.rate_transformer_node(
...     in_size=5,
...     linear_summation=True,
...     input_nonlinearity=sigmoid
... )
>>> transformer.init_state()
>>> # High input drives to saturation
>>> with brainstate.environ.context(dt=0.1 * u.ms):
...     output = transformer.update(instant_rate_events=(10.0, 1.0))
>>> print(output)
[0.9999..., 0.9999..., ...]  # sigmoid(10) ≈ 1.0

Example 3: Delayed event scheduling

>>> import brainpy_state as bst
>>> import saiunit as u
>>> import brainstate
>>> transformer = bst.rate_transformer_node(in_size=3)
>>> transformer.init_state()
>>> # Send delayed event (rate=1.0, weight=0.5, delay=2 steps)
>>> with brainstate.environ.context(dt=1.0 * u.ms):
...     out_t0 = transformer.update(delayed_rate_events=(1.0, 0.5, 2))
...     out_t1 = transformer.update()  # Delayed event still in queue
...     out_t2 = transformer.update()  # Delayed event arrives now
>>> print(out_t0, out_t1, out_t2)
[0., 0., 0.] [0., 0., 0.] [0.5, 0.5, 0.5]  # Event arrives at t+2

Example 4: Per-event nonlinearity (linear_summation=False)

>>> import brainpy_state as bst
>>> import numpy as np
>>> # ReLU applied per-event before summation
>>> relu = lambda h: np.maximum(0, h)
>>> transformer = bst.rate_transformer_node(
...     in_size=1,
...     linear_summation=False,
...     input_nonlinearity=relu
... )
>>> transformer.init_state()
>>> # Two events: one positive, one negative
>>> events = [
...     (2.0, 1.0),   # ReLU(2.0) * 1.0 = 2.0
...     (-1.0, 1.0)   # ReLU(-1.0) * 1.0 = 0.0
... ]
>>> with brainstate.environ.context(dt=0.1 * u.ms):
...     output = transformer.update(instant_rate_events=events)
>>> print(output)
[2.0]  # Sum of per-event transformed values

rate_transformer_node

Contents

rate_transformer_node#