FullyShardedDataParallelStrategy#
- class braintools.trainer.FullyShardedDataParallelStrategy(mesh=None, data_axis='data', model_axis=None)#
Fully Sharded Data Parallel (FSDP) strategy.
Shards both model parameters and gradients across devices, similar to PyTorch’s FSDP.
- Parameters:
Examples
>>> import braintools >>> strategy = braintools.trainer.FullyShardedDataParallelStrategy() >>> strategy.name 'fsdp' >>> strategy.mesh.axis_names ('data',) >>> strategy = braintools.trainer.FullyShardedDataParallelStrategy(model_axis='model') >>> strategy.mesh.axis_names ('data', 'model')