HORNSeqLayer

HORNSeqLayer#

class brainmass.HORNSeqLayer(n_input, n_hidden, alpha=0.04, omega=0.2243994752564138, gamma=0.01, v=0.0, delay_init=ZeroInit(unit=1), x_init=ZeroInit(unit=1), y_init=ZeroInit(unit=1), delay=None, rec_w_init=KaimingNormal(mode=fan_in, nonlinearity=relu, unit=1), rec_b_init=ZeroInit(unit=1), inp_w_init=KaimingNormal(mode=fan_in, nonlinearity=relu, unit=1), inp_b_init=ZeroInit(unit=1))[source]#

Sequential layer wrapper for HORN dynamics with input and recurrent connections.

This layer combines a HORNStep dynamics model with trainable input-to-hidden and hidden-to-hidden (recurrent) linear transformations to process sequential data. It supports optional synaptic delays in recurrent connections and processes entire input sequences using a for-loop scan operation.

The layer computes:

\[\begin{split}\begin{aligned} \mathbf{I}_{t+1}^{\mathrm{ext}} &= \mathbf{W}^{ih} \mathbf{s}_{t+1} + \mathbf{b}^{ih}, \\ \mathbf{I}_{t+1}^{\mathrm{rec}} &= \mathbf{W}^{hh} \mathbf{y}_t + \mathbf{b}^{hh} + \mathbf{v} \cdot \mathbf{x}_t, \\ \mathbf{out}_t &= \text{HORNStep}(\mathbf{I}_{t}^{\mathrm{ext}} + \mathbf{I}_{t}^{\mathrm{rec}}), \end{aligned}\end{split}\]

where \(\mathbf{s}\) is the input sequence, \(\mathbf{W}^{ih}, \mathbf{b}^{ih}\) are input weights/biases, \(\mathbf{W}^{hh}, \mathbf{b}^{hh}\) are recurrent weights/biases, and the output is the position state \(\mathbf{x}\) from the HORN dynamics.

Parameters:

  • n_input (int) – Dimensionality of input features at each time step.

  • n_hidden (int) – Dimensionality of the hidden state (number of HORN oscillators).

  • alpha (Callable | Array | ndarray | bool | number | bool | int | float | complex | Quantity | Param) – Excitability factor for HORN dynamics (dimensionless). Broadcastable to n_hidden. Default is 0.04.

  • omega (Callable | Array | ndarray | bool | number | bool | int | float | complex | Quantity | Param) – Natural frequency for HORN dynamics (radians per time step). Default is 2π/28 ≈ 0.224.

  • gamma (Callable | Array | ndarray | bool | number | bool | int | float | complex | Quantity | Param) – Damping coefficient for HORN dynamics (dimensionless). Default is 0.01.

  • v (Callable | Array | ndarray | bool | number | bool | int | float | complex | Quantity | Param) – Amplitude feedback coefficient (dimensionless). Default is 0.0.

  • x_init (Callable | Array | ndarray | bool | number | bool | int | float | complex | Quantity) – Initializer for HORN position state \(\mathbf{x}\). Default is braintools.init.ZeroInit().

  • y_init (Callable | Array | ndarray | bool | number | bool | int | float | complex | Quantity) – Initializer for HORN velocity state \(\mathbf{y}\). Default is braintools.init.ZeroInit().

  • delay (Array | ndarray | bool | number | bool | int | float | complex | Quantity | Callable | None) – Synaptic delay configuration for recurrent connections. If provided, creates a (n_hidden, n_hidden) delay matrix where each entry specifies the number of time steps to delay that connection. When None, no delays are used. Default is None.

  • rec_w_init (Callable | Array | ndarray | bool | number | bool | int | float | complex | Quantity) – Initializer for recurrent weight matrix \(\mathbf{W}^{hh}\). Default is braintools.init.KaimingNormal().

  • rec_b_init (Array | ndarray | bool | number | bool | int | float | complex | Quantity | Callable | None) – Initializer for recurrent bias vector \(\mathbf{b}^{hh}\). If None, no recurrent bias is used. Default is braintools.init.ZeroInit().

  • inp_w_init (Callable | Array | ndarray | bool | number | bool | int | float | complex | Quantity) – Initializer for input weight matrix \(\mathbf{W}^{ih}\). Default is braintools.init.KaimingNormal().

  • inp_b_init (Array | ndarray | bool | number | bool | int | float | complex | Quantity | Callable | None) – Initializer for input bias vector \(\mathbf{b}^{ih}\). If None, no input bias is used. Default is braintools.init.ZeroInit().

Return type:

Any

horn#

The underlying HORN dynamics model instance.

Type:

HORNStep

i2h#

Input-to-hidden linear transformation with shape (n_input, n_hidden).

Type:

brainstate.nn.Linear

h2h#

Hidden-to-hidden recurrent transformation. When delay is None, this is a standard Linear layer with shape (n_hidden, n_hidden). When delay is specified, this is an AdditiveCoupling operator that applies delayed synaptic connections.

Type:

brainstate.nn.Linear or AdditiveCoupling

Notes

  • Sequential processing: The update method uses brainstate.transform.for_loop to scan over the time dimension of the input sequence, maintaining the HORN hidden states across time steps.

  • Delay implementation: When synaptic delays are specified via the delay parameter, the layer uses an AdditiveCoupling operator that prefetches delayed values of the velocity state \(\mathbf{y}\) from a ring buffer. This enables modeling of axonal/dendritic delays in recurrent connections.

  • State recording: The record_state parameter in the update method allows recording both position and velocity states at each time step, returning them alongside the output sequence.

  • Parameter sharing: All HORN dynamics parameters (alpha, omega, gamma, v, h) are shared across all oscillators in the layer, though they can be initialized as arrays to provide per-oscillator heterogeneity.

See also

HORNStep

Single-step HORN dynamics

HORNSeqNetwork

Multi-layer HORN network