Tutorial 3: Variable-length trial sequences#

Real cognitive experiments rarely use a single fixed timeline. Delay periods are jittered, trials may branch on a rule, and some loops stop early. This tutorial shows the current variable-length support in braintools.cogtask.

The framework uses a packed-buffer + mask design:

  • Every trial is written into a buffer sized to the worst-case length, task.max_trial_duration().

  • sample_trial returns info['mask'], a boolean (T_max,) array marking the valid timesteps.

  • batch_sample(..., return_mask=True) stacks masks alongside X and Y.

  • Trailing padding stays zero in X/Y and False in mask.

Current JIT/vmap status:

  • VariableDuration, If, and integer-keyed Switch work with batch_sample.

  • While works for eager sample_trial when its condition is concrete. A data-dependent traced While condition raises NotImplementedError under batch_sample; use VariableDuration or bounded Repeat for that case.

import numpy as np
import brainstate
import brainunit as u
import jax
import jax.numpy as jnp

from braintools.cogtask import (
    Task,
    Feature,
    FeatureSet,
    concat,
    Fixation,
    Delay,
    Response,
    Stimulus,
    Sample,
    Test,
    Cue,
    VariableDuration,
    If,
    Switch,
    While,
    TruncExp,
    UniformDuration,
    label,
    phase_tree_is_variable,
)

DT = 1.0 * u.ms
brainstate.environ.set(dt=DT)

1. VariableDuration#

VariableDuration is the declarative phase for a trial-dependent duration. It looks like any other phase, with inputs=, outputs=, and noise=, but its actual length is read from trial state:

  • ctx_key names the entry that trial_init writes.

  • max_duration is the static upper bound used to size the packed slot.

  • min_duration documents the sampler’s lower bound. It is not enforced, so trial_init should sample values inside [min_duration, max_duration].

  • The realized value may be a brainunit Quantity in any time unit, or a plain number in milliseconds. It is divided by dt, truncated to integer steps, and clipped to [1, slot size].

Pass dt= to the task constructor when using batch_sample, so the static buffer shape belongs to the task object.

variable_delay = VariableDuration(
    min_duration=200 * u.ms,
    max_duration=1500 * u.ms,
    ctx_key='delay_duration',
    inputs={'fixation': 1.0},
    outputs={'label': 0},
    name='variable_delay',
)

variable_delay
DeclarativePhase('variable_delay', inputs=['fixation'])

2. A minimal variable-delay task#

This task has fixed fixation, sample, and response phases, plus one delay whose length is sampled independently for each trial.

fix = Feature(1, 'fixation')
stim = Feature(2, 'stim')
choice = Feature(2, 'choice')

phases = concat([
    Fixation(50 * u.ms, inputs={'fixation': 1.0}, outputs={'label': 0}),
    Sample(
        40 * u.ms,
        inputs={'stim': lambda ctx, feature: jnp.ones(feature.num)},
        outputs={'label': 0},
    ),
    VariableDuration(
        min_duration=200 * u.ms,
        max_duration=1500 * u.ms,
        ctx_key='delay_duration',
        inputs={'fixation': 1.0},
        outputs={'label': 0},
        name='delay',
    ),
    Response(50 * u.ms, outputs={'label': label('gt')}),
])


def init(ctx):
    ctx['delay_duration'] = ctx.rng.uniform(200.0, 1500.0)  # plain float = ms
    ctx['gt'] = ctx.rng.choice(2).astype(jnp.int32) + 1


task = Task(
    phases=phases,
    input_features=fix + stim,
    output_features=fix + choice,
    trial_init=init,
    seed=0,
    num_classes=3,
    dt=DT,
)

task
Task(name=Task, inputs=3, outputs=3, output_mode=categorical)

3. Inspect the packed shape#

Task detects variable-length phase trees at construction time. For packed sampling, task.max_trial_duration() is the static T_max used by both sample_trial and batch_sample.

print('is_variable_length    :', task.is_variable_length)
print('max_trial_duration()  :', task.max_trial_duration(), 'steps')
print('phase_tree_is_variable:', phase_tree_is_variable(phases))
print('expected T_max         :', 50 + 40 + 1500 + 50)
is_variable_length    : True
max_trial_duration()  : 1640 steps
phase_tree_is_variable: True
expected T_max         : 1640

4. Sample with masks#

For variable-length tasks, sample_trial(index) returns (X, Y, info) where info['mask'] marks the valid prefix of the packed buffer. For fixed-length tasks, info['mask'] is None, but batch_sample(..., return_mask=True) can still return an all-True mask.

X, Y, info = task.sample_trial(0)
mask = info['mask']

print('X.shape    =', X.shape, '(T_max, num_inputs)')
print('Y.shape    =', Y.shape, '(T_max,)')
print('mask.shape =', mask.shape, 'mask.dtype =', mask.dtype)
print('valid steps for trial 0 =', int(np.asarray(mask).sum()))
print('full T_max              =', X.shape[0])
X.shape    = (1640, 3) (T_max, num_inputs)
Y.shape    = (1640,) (T_max,)
mask.shape = (1640,) mask.dtype = bool
valid steps for trial 0 = 475
full T_max              = 1640
X, Y, mask = task.batch_sample(8, return_mask=True)
print('time-first X.shape    =', X.shape)
print('time-first Y.shape    =', Y.shape)
print('time-first mask.shape =', mask.shape)
print('per-trial valid lengths =', np.asarray(mask.sum(axis=0)))

X_bt, Y_bt, mask_bt = task.batch_sample(8, time_first=False, return_mask=True)
print('batch-first X.shape   =', X_bt.shape)
print('batch-first mask sums =', np.asarray(mask_bt.sum(axis=1)))
time-first X.shape    = (1640, 8, 3)
time-first Y.shape    = (1640, 8)
time-first mask.shape = (1640, 8)
per-trial valid lengths = [ 475  458 1309 1443 1132 1636  827  358]
batch-first X.shape   = (8, 1640, 3)
batch-first mask sums = [ 475  458 1309 1443 1132 1636  827  358]

Shape contract#

Call

Returned shapes

task.sample_trial(i)

X: (T_max, F), Y: (T_max,) or (T_max, F_out), info['mask']: (T_max,)

task.batch_sample(B)

X: (T_max, B, F), Y: (T_max, B) or (T_max, B, F_out)

task.batch_sample(B, return_mask=True)

adds mask: (T_max, B)

task.batch_sample(B, time_first=False, return_mask=True)

X: (B, T_max, F), Y: (B, T_max), mask: (B, T_max)

Trailing positions stay zero#

Packed buffers store each realized trial as a valid prefix. Everything after that prefix is padding.

trial_idx = int(np.argmin(np.asarray(mask.sum(axis=0))))
valid = int(np.asarray(mask[:, trial_idx]).sum())

print(f'trial {trial_idx}: {valid} valid steps, {X.shape[0] - valid} padded steps')
print('tail of X  :', X[valid:valid + 4, trial_idx])
print('tail of Y  :', Y[valid:valid + 4, trial_idx])
print('tail mask  :', mask[valid:valid + 4, trial_idx])
trial 7: 358 valid steps, 1282 padded steps
tail of X  : [[0. 0. 0.]
 [0. 0. 0.]
 [0. 0. 0.]
 [0. 0. 0.]]
tail of Y  : [0 0 0 0]
tail mask  : [False False False False]
fixed_task = Task(
    phases=concat([
        Fixation(10 * u.ms, inputs={'fixation': 1.0}, outputs={'label': 0}),
        Response(5 * u.ms, outputs={'label': 1}),
    ]),
    input_features=FeatureSet(fix),
    output_features=fix + choice,
    num_classes=3,
    dt=DT,
)

_, _, fixed_mask = fixed_task.batch_sample(3, return_mask=True)
print('fixed mask shape =', fixed_mask.shape)
print('fixed mask all True?', bool(np.asarray(fixed_mask).all()))
fixed mask shape = (15, 3)
fixed mask all True? True

5. Conditional control flow#

If, Switch, and While are compound phases, so any task containing them is handled by the packed path.

Phase

Current behavior

If

Uses jax.lax.cond under batch_sample; both branches must have compatible buffer pytrees. The mask records the selected branch length.

Switch

Concrete selectors can use hashable keys. Traced selectors under batch_sample must use integer case keys and dispatch through jax.lax.switch.

While

Eager sample_trial works when the condition is a concrete Python bool. Data-dependent traced conditions are not supported under batch_sample.

if_phases = concat([
    Fixation(20 * u.ms, inputs={'fixation': 1.0}, outputs={'label': 0}),
    If(
        condition=lambda ctx: ctx['go'],
        then=Stimulus(
            60 * u.ms,
            name='go_stimulus',
            inputs={'stim': lambda ctx, feature: jnp.ones(feature.num)},
            outputs={'label': 1},
        ),
        else_=Delay(
            20 * u.ms,
            name='nogo_delay',
            inputs={'fixation': 0.5},
            outputs={'label': 2},
        ),
    ),
    Response(20 * u.ms, outputs={'label': label('gt')}),
])


def if_init(ctx):
    go = (jnp.asarray(ctx['trial_index']) % 2) == 0
    ctx['go'] = go
    ctx['gt'] = jnp.where(go, 1, 2).astype(jnp.int32)


if_task = Task(
    phases=if_phases,
    input_features=fix + stim,
    output_features=fix + choice,
    trial_init=if_init,
    seed=0,
    num_classes=3,
    dt=DT,
)

X_if, Y_if, mask_if = if_task.batch_sample(6, return_mask=True)
last_if_labels = [int(Y_if[int(mask_if[:, i].sum()) - 1, i]) for i in range(6)]
print('T_max =', if_task.max_trial_duration())
print('valid steps per trial =', np.asarray(mask_if.sum(axis=0)))
print('last valid labels =', last_if_labels)
T_max = 100
valid steps per trial = [100  60 100  60 100  60]
last valid labels = [1, 2, 1, 2, 1, 2]
switch_task = Task(
    phases=concat([
        Fixation(20 * u.ms, inputs={'fixation': 1.0}, outputs={'label': 0}),
        Switch(
            selector=lambda ctx: jnp.asarray(ctx['rule'], dtype=jnp.int32),
            cases={
                0: Cue(
                    30 * u.ms,
                    name='rule_0',
                    inputs={'rule': 1.0},
                    outputs={'label': 1},
                ),
                1: Cue(
                    60 * u.ms,
                    name='rule_1',
                    inputs={'rule': 1.0},
                    outputs={'label': 2},
                ),
                2: Cue(
                    90 * u.ms,
                    name='rule_2',
                    inputs={'rule': 1.0},
                    outputs={'label': 3},
                ),
            },
        ),
    ]),
    input_features=Feature(1, 'fixation') + Feature(3, 'rule'),
    output_features=Feature(1, 'fixation') + Feature(3, 'choice'),
    trial_init=lambda ctx: ctx.update(rule=jnp.asarray(ctx['trial_index']) % 3),
    seed=0,
    num_classes=4,
    dt=DT,
)

X_sw, Y_sw, mask_sw = switch_task.batch_sample(6, return_mask=True)
last_sw_labels = [int(Y_sw[int(mask_sw[:, i].sum()) - 1, i]) for i in range(6)]
print('valid steps per trial =', np.asarray(mask_sw.sum(axis=0)))
print('last valid labels =', last_sw_labels)
valid steps per trial = [ 50  80 110  50  80 110]
last valid labels = [1, 2, 3, 1, 2, 3]
def increment_counter(ctx):
    ctx['counter'] = int(ctx.get('counter', 0)) + 1


while_task = Task(
    phases=While(
        condition=lambda ctx: int(ctx.get('counter', 0)) < ctx['target_count'],
        body=Stimulus(
            25 * u.ms,
            inputs={'stim': 1.0},
            outputs={'label': 1},
            on_exit=increment_counter,
        ),
        max_iterations=5,
    ),
    input_features=Feature(1, 'stim'),
    output_features=Feature(1, 'choice'),
    trial_init=lambda ctx: ctx.update(counter=0, target_count=3),
    seed=0,
    num_classes=2,
    dt=DT,
)

X_while, Y_while, info_while = while_task.sample_trial(0)
print('allocated steps =', X_while.shape[0])
print('valid steps =', int(np.asarray(info_while['mask']).sum()))
print('iterations =', info_while['trial_state']['while_total_iterations'])
allocated steps = 125
valid steps = 75
iterations = 3
def add_evidence(ctx):
    ctx['evidence'] = ctx.get('evidence', 0.0) + 0.4


traced_while_task = Task(
    phases=While(
        condition=lambda ctx: ctx.get('evidence', 0.0) < ctx['threshold'],
        body=Stimulus(
            10 * u.ms,
            inputs={'stim': 1.0},
            outputs={'label': 1},
            on_exit=add_evidence,
        ),
        max_iterations=10,
    ),
    input_features=Feature(1, 'stim'),
    output_features=Feature(1, 'choice'),
    trial_init=lambda ctx: ctx.update(
        evidence=0.0,
        threshold=ctx.rng.uniform(0.5, 1.5),
    ),
    seed=0,
    num_classes=2,
    dt=DT,
)

try:
    traced_while_task.batch_sample(4, return_mask=True)
except NotImplementedError as exc:
    print(type(exc).__name__ + ': data-dependent While conditions are eager-only under batch_sample')
NotImplementedError: data-dependent While conditions are eager-only under batch_sample

6. Migrated built-in tasks#

These pre-built tasks currently use VariableDuration internally and work with batch_sample(..., return_mask=True):

  • HierarchicalReasoning: variable delay between flash cues.

  • IntervalDiscrimination: two independently sampled intervals.

  • ReadySetGo: sampled measurement interval.

from braintools.cogtask import HierarchicalReasoning, IntervalDiscrimination, ReadySetGo

for cls in [HierarchicalReasoning, IntervalDiscrimination, ReadySetGo]:
    builtin_task = cls(seed=42, dt=DT)
    X_builtin, Y_builtin, M_builtin = builtin_task.batch_sample(4, return_mask=True)
    print(
        f'{cls.__name__:25s} T_max={builtin_task.max_trial_duration():5d}'
        f'  variable={builtin_task.is_variable_length}'
        f'  mask sums={np.asarray(M_builtin.sum(axis=0)).tolist()}'
    )
HierarchicalReasoning     T_max= 2000  variable=True  mask sums=[1649, 1489, 1537, 1862]
IntervalDiscrimination    T_max= 3100  variable=True  mask sums=[2604, 2634, 2646, 2778]
ReadySetGo                T_max= 2800  variable=True  mask sums=[2566, 2459, 2491, 2708]

7. Masked losses and metrics#

A packed batch should normally weight losses and metrics by the mask. For categorical cross entropy, compute the per-timestep loss first, multiply by the mask, and normalize by mask.sum() instead of the full buffer size.

def masked_cross_entropy(logits, labels, mask):
    # logits: (T, B, C), labels: (T, B), mask: (T, B)
    logp = jax.nn.log_softmax(logits, axis=-1)
    onehot = jax.nn.one_hot(labels, logp.shape[-1])
    nll = -jnp.sum(onehot * logp, axis=-1)
    mask_f = mask.astype(nll.dtype)
    return jnp.sum(nll * mask_f) / jnp.maximum(jnp.sum(mask_f), 1.0)


X_loss, Y_loss, M_loss = task.batch_sample(4, return_mask=True)
logits = jnp.zeros(X_loss.shape[:2] + (task.num_classes,))
print('masked loss =', float(masked_cross_entropy(logits, Y_loss, M_loss)))
masked loss = 1.0986124277114868

The same pattern applies to vector-output tasks: compute an elementwise vector loss, multiply by mask[..., None], and normalize by the number of valid steps. Accuracy can use (preds == Y) * mask divided by mask.sum().

8. Duration samplers#

TruncExp and UniformDuration are helpers for drawing per-trial durations in trial_init. They consume ctx.rng when called as sampler(ctx), so they are compatible with seeded sample_trial and batched sampling.

Their min_value() and max_value() methods expose the bounds you should pass to VariableDuration.

te = TruncExp(mean=600 * u.ms, min_val=300 * u.ms, max_val=1500 * u.ms)
ud = UniformDuration(200 * u.ms, 800 * u.ms)

print('TruncExp        bounds:', te.min_value(), '-', te.max_value(), 'is_variable =', te.is_variable)
print('UniformDuration bounds:', ud.min_value(), '-', ud.max_value(), 'is_variable =', ud.is_variable)
TruncExp        bounds: 300 ms - 1500 ms is_variable = True
UniformDuration bounds: 200 ms - 800 ms is_variable = True
delay_dist = TruncExp(mean=600 * u.ms, min_val=300 * u.ms, max_val=1500 * u.ms)


def init_with_sampler(ctx):
    ctx['delay_duration'] = delay_dist(ctx)  # a Quantity; converted using dt
    ctx['gt'] = ctx.rng.choice(2).astype(jnp.int32) + 1


sampler_task = Task(
    phases=concat([
        Fixation(50 * u.ms, inputs={'fixation': 1.0}, outputs={'label': 0}),
        VariableDuration(
            min_duration=delay_dist.min_value(),
            max_duration=delay_dist.max_value(),
            ctx_key='delay_duration',
            inputs={'fixation': 1.0},
            outputs={'label': 0},
        ),
        Response(50 * u.ms, outputs={'label': label('gt')}),
    ]),
    input_features=fix + stim,
    output_features=fix + choice,
    trial_init=init_with_sampler,
    seed=0,
    num_classes=3,
    dt=DT,
)

X_sampled, Y_sampled, M_sampled = sampler_task.batch_sample(8, return_mask=True)
print('T_max =', sampler_task.max_trial_duration())
print('per-trial lengths =', np.asarray(M_sampled.sum(axis=0)).tolist())
T_max = 1600
per-trial lengths = [456, 449, 1020, 1194, 849, 1588, 635, 407]

Summary#

  • Use VariableDuration(min_duration, max_duration, ctx_key=...) for epochs whose length is sampled per trial.

  • Write the realized duration into ctx[ctx_key] from trial_init, inside the declared bounds, either as a Quantity or as a plain number of milliseconds.

  • Use task.is_variable_length and task.max_trial_duration() to inspect the packed buffer contract.

  • Use batch_sample(B, return_mask=True) and weight losses or metrics by the returned mask.

  • If and integer-keyed Switch are batch-friendly; data-dependent While currently remains an eager sample_trial pattern.

For the full API surface, see the braintools.cogtask API reference.