MultiSensoryIntegration

MultiSensoryIntegration#

class braintools.cogtask.MultiSensoryIntegration(t_fixation=Quantity(300., 'ms'), t_stimulus=Quantity(750., 'ms'), t_decision=Quantity(100., 'ms'), num_choices=2, coherences=(5.0, 15.0, 50.0), noise_sigma=Quantity(1., 'ms^0.5'), pop_per_choice=1, p_unisensory=0.0, **kwargs)[source]#

Multi-sensory integration task.

Two-alternative (or num_choices-alternative) forced choice in which the motion direction is conveyed through two sensory modalities at once, as in the visual/auditory rate-discrimination paradigm of Raposo et al. [1]. Both modalities point toward the same (correct) direction; the trial’s total coherence coh is split between them, so the evidence of the two modalities has to be combined to reach the best decision. This is the MultiSensoryIntegration environment of NeuroGym [2].

Structure: Fixation >> Stimulus >> Decision

Parameters:

  • t_fixation (Quantity) – Fixation period (default: 300 ms, as in NeuroGym).

  • t_stimulus (Quantity) – Stimulus period, during which both modalities are shown (default: 750 ms).

  • t_decision (Quantity) – Decision period (default: 100 ms).

  • num_choices (int) – Number of choice directions, NeuroGym’s dim_ring (default: 2). Must be at least 2.

  • coherences (Sequence[float]) – Total coherence levels in [0, 100] from which coh is drawn (default: (5, 15, 50)).

  • noise_sigma (Array | ndarray | bool | number | bool | int | float | complex | Quantity) – Standard deviation of the Gaussian input noise added to both modality channels during the stimulus, in units of ms ** 0.5; it is scaled by 1 / sqrt(dt) (default: 1.0 * u.ms ** 0.5).

  • pop_per_choice (int) – Number of cosine-tuned units per choice in each modality (default: 1, which reproduces NeuroGym’s observation space).

  • p_unisensory (float) – Probability in [0, 1] that a trial is unisensory: only one of the two modalities (chosen with equal probability) is shown and carries the whole coherence, while the other channel is silent (zero input plus noise). The default 0.0 gives NeuroGym’s always-multisensory trials.

  • **kwargs – Forwarded to Task (e.g. seed, dt, output_mode).

Raises:

ValueError – If num_choices < 2, pop_per_choice < 1, coherences is empty or p_unisensory is outside [0, 1].

See also

ContextDecisionMaking

Two modalities, but a context cue selects one.

PerceptualDecisionMaking

A single noisy motion stimulus.

Notes

Inputs (1 + 2 * num_choices * pop_per_choice channels): fixation (1), modality1 and modality2. Unit i of a modality with preferred direction \(\phi_i\) receives

\[x_i = 0.5 + \frac{c_m}{200} \cos(\phi_i - \theta^*)\]

where \(\theta^*\) is the correct direction and \(c_m\) the modality’s coherence. Multisensory trials draw a split coh_prop \(\sim U(0, 1)\) and set \(c_1 = coh \cdot coh\_prop\) and \(c_2 = coh \cdot (1 - coh\_prop)\), so \(c_1 + c_2 = coh\). The fixation input is 1 during fixation and stimulus and 0 during the decision period.

Labels (categorical): 0 (fixate) during fixation and stimulus, ground_truth + 1 during the decision period; num_classes is 1 + num_choices.

Trial state: ground_truth, coherence (total), coh_prop, mod1_coherence, mod2_coherence, modality_condition (0 = both, 1 = only modality 1, 2 = only modality 2), mod1_present, mod2_present and stimulus_direction.

Differences from NeuroGym. Phase names follow this package ('decision' is a Response phase); pop_per_choice and p_unisensory are additions (their defaults reproduce NeuroGym); noise is added to the two modality channels only, not to the fixation channel; there is no reward or abort signal, the task is supervised through the labels.

Trials are generated with JAX, so batch_sample runs under jax.jit and vmap.

References

Examples

>>> import brainunit as u
>>> import braintools
>>> task = braintools.cogtask.MultiSensoryIntegration(seed=0, dt=1.0 * u.ms)
>>> task.num_inputs, task.num_outputs
(5, 3)
>>> X, Y, info = task.sample_trial(0)
>>> X.shape, Y.shape
((1150, 5), (1150,))
>>> state = info['trial_state']
>>> int(Y[-1]) == int(state['ground_truth']) + 1
True
>>> bool(abs(state['mod1_coherence'] + state['mod2_coherence']
...          - state['coherence']) < 1e-4)
True
>>>
>>> X, Y = task.batch_sample(4)
>>> X.shape, Y.shape
((1150, 4, 5), (1150, 4))
define_features()[source]#

Define input and output features.

Override in subclass for class-based task definition.

Return type:

Tuple

Returns:

  • input_features (Feature or FeatureSet) – Input feature definitions.

  • output_features (Feature or FeatureSet) – Output feature definitions.

define_phases()[source]#

Define the phase structure.

Override in subclass for class-based task definition.

Returns:

The task phase structure (single phase or composition).

Return type:

Phase

trial_init(ctx)[source]#

Initialize trial-level state.

Override in subclass to set up trial parameters like ground_truth, stimulus indices, etc.

Parameters:

ctx (Context) – Trial context to populate with state.

Return type:

None