braintools.trainer.create_distributed_batches#
- braintools.trainer.create_distributed_batches(data, batch_size, devices=None, shuffle=False, seed=None)[source]#
Create batches suitable for distributed training with pmap.
- Parameters:
- Yields:
Any – Batches with shape (num_devices, batch_size, …).
Notes
Only full blocks of
num_devices * batch_sizesamples are yielded; any trailing samples that cannot fill a complete block are dropped (this keeps the leading device axis a fixed size forpmap). Size the data orbatch_sizeaccordingly if you cannot afford to drop the remainder.Examples
Batches carry a leading device axis, ready for
jax.pmap:>>> import jax >>> import jax.numpy as jnp >>> import braintools >>> X = jnp.ones((10, 3)) >>> devices = jax.devices()[:1] >>> batches = list(braintools.trainer.create_distributed_batches( ... X, batch_size=4, devices=devices)) >>> len(batches) # the trailing 2 samples cannot fill a block 2 >>> batches[0].shape (1, 4, 3)