brainstate.transform.vmap2_new_states#
- brainstate.transform.vmap2_new_states(module, init_kwargs, state_tag=None, axis_size=None, state_out_axes=None, spmd_axis_name=None)#
Initialize and vectorize newly created states within a module.
Wraps
module.init_all_states(**init_kwargs)in avmap2()-style transform, executes itaxis_sizetimes in parallel, and restores the vectorized values back onto the freshly created state objects. Random states are split per lane, so random initializers produce a distinct draw for every batch member.Note
init_all_statesruns twice (a probe pass plus the mapped pass) and must be idempotent; see_map_new_states()for details.- Parameters:
module (Module) – Module whose
init_all_statescreates the states to vectorize.init_kwargs (
Dict) – Keyword arguments forwarded tomodule.init_all_states.state_tag (
str) – Tag applied to the newly created states.axis_size (
int) – Size of the vectorization axis. Required.state_out_axes (
Dict[int,type|str|Callable[[Tuple[Key,...],Any],bool] |bool|EllipsisType|None|Tuple[Filter,...] |List[Filter]]) – Output-axis selectors.None(default) batches every state on axis0exceptNonBatchState, which is replicated on axisNone.spmd_axis_name (
Hashable|Tuple[Hashable,...] |None) – SPMD axis label forwarded to the underlyingjax.vmap.
- Returns:
Mapping of axis identifier to the lists of vectorized states.
- Return type:
- Raises:
ValueError – If
axis_sizeis not provided.
See also
brainstate.transform.vmap2Vectorize a callable with state semantics.
brainstate.transform.pmap2_new_statesMulti-device variant.
brainstate.NonBatchStateMarker for states that should be replicated.
Examples
>>> import brainstate >>> import jax.numpy as jnp >>> >>> class Counter(brainstate.nn.Module): ... def init_state(self): ... self.count = brainstate.ShortTermState(jnp.zeros(())) >>> >>> module = Counter() >>> _ = brainstate.transform.vmap2_new_states( ... module, init_kwargs={}, axis_size=5 ... ) >>> module.count.value.shape (5,)