HORNSeqNetwork

class brainmass.HORNSeqNetwork(n_input, n_hidden, n_output, alpha=0.04, omega=0.2243994752564138, gamma=0.01, v=0.0, x_init=ZeroInit(unit=1), y_init=ZeroInit(unit=1), delay_init=ZeroInit(unit=1), delay=None, rec_w_init=KaimingNormal(mode=fan_in, nonlinearity=relu, unit=1), rec_b_init=ZeroInit(unit=1), inp_w_init=KaimingNormal(mode=fan_in, nonlinearity=relu, unit=1), inp_b_init=ZeroInit(unit=1))

Multi-layer HORN network for sequential processing tasks.

This network stacks multiple HORNSeqLayer instances to create a deep recurrent architecture based on harmonic oscillator dynamics. It is designed for sequence-to-sequence or sequence-to-label tasks such as time series classification, sequence generation, or temporal pattern recognition.

The network processes input sequences through L layers of HORN dynamics:

\[\begin{split}\begin{aligned} \mathbf{h}^{(0)}_t &= \mathbf{s}_t, \\ \mathbf{h}^{(\ell)}_t &= \text{HORNSeqLayer}^{(\ell)}(\mathbf{h}^{(\ell-1)}), \quad \ell = 1, \ldots, L, \\ \mathbf{o}_T &= \mathbf{W}^{ho} \mathbf{x}^{(L)}_T + \mathbf{b}^{ho}, \end{aligned}\end{split}\]

where \(\mathbf{s}\) is the input sequence, \(\mathbf{h}^{(\ell)}\) denotes the hidden state sequence at layer \(\ell\), \(\mathbf{x}^{(L)}_T\) is the final position state from the last layer at the final time step, and \(\mathbf{o}_T\) is the output.

Parameters:

• n_input (int) – Dimensionality of input features at each time step.

• n_hidden (int | Sequence[int]) – Hidden state dimensionality for each layer. If an integer, creates a single hidden layer with that size. If a sequence (list or tuple), creates multiple layers where each element specifies the hidden size for that layer.

• n_output (int) – Dimensionality of the final output.

• alpha (Callable | Array | ndarray | bool | number | bool | int | float | complex | Quantity | Param) – Excitability factor for HORN dynamics (dimensionless), shared across all layers. Default is 0.04.

• omega (Callable | Array | ndarray | bool | number | bool | int | float | complex | Quantity | Param) – Natural frequency for HORN dynamics (radians per time step), shared across all layers. Default is 2π/28 ≈ 0.224.

• gamma (Callable | Array | ndarray | bool | number | bool | int | float | complex | Quantity | Param) – Damping coefficient for HORN dynamics (dimensionless), shared across all layers. Default is 0.01.

• v (Callable | Array | ndarray | bool | number | bool | int | float | complex | Quantity | Param) – Amplitude feedback coefficient (dimensionless), shared across all layers. Default is 0.0.

• x_init (Callable | Array | ndarray | bool | number | bool | int | float | complex | Quantity) – Initializer for HORN position state \(\mathbf{x}\) in all layers. Default is braintools.init.ZeroInit().

• y_init (Callable | Array | ndarray | bool | number | bool | int | float | complex | Quantity) – Initializer for HORN velocity state \(\mathbf{y}\) in all layers. Default is braintools.init.ZeroInit().

• delay (Array | ndarray | bool | number | bool | int | float | complex | Quantity | Callable | None) – Synaptic delay configuration for recurrent connections in all layers. If None, no delays are used. Default is None.

• rec_w_init (Callable | Array | ndarray | bool | number | bool | int | float | complex | Quantity) – Initializer for recurrent weight matrices in all layers. Default is braintools.init.KaimingNormal().

• rec_b_init (Array | ndarray | bool | number | bool | int | float | complex | Quantity | Callable | None) – Initializer for recurrent bias vectors in all layers. Default is braintools.init.ZeroInit().

• inp_w_init (Callable | Array | ndarray | bool | number | bool | int | float | complex | Quantity) – Initializer for input weight matrices (layer-to-layer and final output). Default is braintools.init.KaimingNormal().

• inp_b_init (Array | ndarray | bool | number | bool | int | float | complex | Quantity | Callable | None) – Initializer for input bias vectors (layer-to-layer and final output). Default is braintools.init.ZeroInit().

Return type:

Any

layers

List of HORN sequential layers. The number of layers equals the length of n_hidden if it is a sequence, or 1 if n_hidden is an integer.

Type:

list of HORNSeqLayer

h2o

Hidden-to-output linear transformation mapping the final layer’s position state to the output space. Shape is (n_hidden[-1], n_output).

Type:

brainstate.nn.Linear

Notes

• Layer stacking: The output of layer \(\ell\) (the position state sequence \(\mathbf{x}^{(\ell)}\)) becomes the input to layer \(\ell+1\). Each layer maintains its own position and velocity states.

• Final output: The update method returns the output of the hidden-to-output transformation applied to the position state \(\mathbf{x}^{(L)}_T\) from the last HORN layer at the final time step. This is suitable for sequence classification tasks.

• Prediction mode: The predict method returns the position state sequences from all layers, which can be useful for analyzing the hierarchical representations learned by the network.

• Parameter sharing: All HORN dynamics parameters (alpha, omega, gamma, v, h) are shared across all layers and all oscillators within each layer. This reduces the number of hyperparameters but can be relaxed by passing arrays or per-layer configurations.

• Flexible depth: The network depth can be easily configured by adjusting the n_hidden parameter. For example, n_hidden=[64, 128, 64] creates a 3-layer network with varying hidden sizes.

See also

HORNStep

Single-step HORN dynamics

HORNSeqLayer

Single-layer HORN sequential processor

Examples

Create a 2-layer HORN network for sequence classification:

>>> import brainstate
>>> net = HORNSeqNetwork(
...     n_input=10,
...     n_hidden=[64, 128],
...     n_output=5,
...     alpha=0.04,
...     omega=2*3.14159/28
... )
>>> inputs = brainstate.random.randn(20, 10)  # (time_steps, input_dim)
>>> output = net(inputs)  # (output_dim,)

update(inputs)

Process input sequence through all layers and return final output.

Sequentially processes the input through each HORN layer, then applies a linear transformation to the final layer’s position state to produce the output. Suitable for sequence classification or regression tasks.

Parameters:

inputs (array-like) – Input sequence with shape (T, batch?, n_input) where T is the number of time steps, batch? represents optional batch dimensions, and n_input is the input feature dimension.

Returns:

Final output with shape (n_output,) or (batch?, n_output). Computed by applying the hidden-to-output transformation h2o to the position state of the last layer.

Return type:

array-like

Notes

This method performs the following steps:

1. Process input through layer 1 to get hidden sequence 1

2. Process hidden sequence 1 through layer 2 to get hidden sequence 2

3. Continue through all remaining layers

4. Apply linear transformation to final layer’s position state

The output uses only the final position state x from the last layer, not the full sequence.

hidden_activation(inputs)

Process input sequence and return outputs from all layers.

Similar to update, but returns the position state sequences from all intermediate layers. Useful for analyzing hierarchical representations or multi-scale temporal features learned by the network.

Parameters:

inputs (array-like) – Input sequence with shape (T, batch?, n_input) where T is the number of time steps, batch? represents optional batch dimensions, and n_input is the input feature dimension.

Returns:

List of position state sequences, one per layer. Each element has shape (T, batch?, n_hidden[i]) where n_hidden[i] is the hidden size of layer i. The list length equals the number of layers.

Return type:

list of array-like

Notes

Unlike update, this method does not apply the h2o transformation to produce a final output. Instead, it returns the raw position state sequences from each layer, which can be useful for:

• Visualizing layer-wise dynamics

• Extracting multi-scale temporal features

• Analyzing information flow through the network

• Debugging layer activations

See also

update

Standard forward pass for task output