DistributedSampler#
- class braintools.trainer.DistributedSampler(data_source, num_replicas=None, rank=None, shuffle=True, seed=0, drop_last=False)#
Sampler for distributed training.
Splits data across multiple replicas (devices/processes).
- Parameters:
Examples
Each replica gets an interleaved 1/4 of the data:
>>> import jax.numpy as jnp >>> import braintools >>> dataset = braintools.trainer.ArrayDataset(jnp.arange(8)) >>> sampler = braintools.trainer.DistributedSampler( ... dataset, num_replicas=4, rank=1, shuffle=False) >>> list(sampler) [1, 5]