DistributedDataLoader#
- class braintools.trainer.DistributedDataLoader(dataset, batch_size=32, num_replicas=None, rank=None, shuffle=True, seed=0, drop_last=False, **kwargs)#
DataLoader for distributed training.
Automatically shards data across multiple devices/processes.
- Parameters:
batch_size (
int) – Batch size per replica.num_replicas (
int|None) – Number of replicas. Default is number of JAX devices.shuffle (
bool) – Whether to shuffle data.seed (
int) – Random seed for shuffling.drop_last (
bool) – Whether to drop incomplete batches.**kwargs – Additional arguments passed to DataLoader.
Examples
Each replica iterates over its own shard of the data:
>>> import jax.numpy as jnp >>> import braintools >>> X = jnp.arange(16.0).reshape(8, 2) >>> loader = braintools.trainer.DistributedDataLoader( ... X, batch_size=2, num_replicas=2, rank=0, shuffle=False) >>> len(loader) 2 >>> [batch[0][:, 0].tolist() for batch in loader] [[0.0, 4.0], [8.0, 12.0]]