Trainer#
- class braintools.trainer.Trainer(max_epochs=1000, min_epochs=1, max_steps=-1, val_check_interval=1.0, check_val_every_n_epoch=1, callbacks=None, logger=True, enable_progress_bar=True, enable_checkpointing=True, default_root_dir=None, gradient_clip_val=None, gradient_clip_algorithm='norm', accumulate_grad_batches=1, devices='auto', strategy='auto', precision='32', deterministic=False, benchmark=False, seed=None)#
Orchestrates the training process.
The Trainer handles the training loop, validation, testing, and prediction, integrating callbacks, logging, checkpointing, and distributed training.
- Parameters:
max_epochs (
int) – Maximum number of training epochs.min_epochs (
int) – Minimum number of training epochs.max_steps (
int) – Maximum number of optimizer steps. -1 means no limit. Whenaccumulate_grad_batches > 1one step spans that many batches.val_check_interval (
int|float) – How often to run validation within a training epoch. Integer = every N batches, float = fraction of epoch.check_val_every_n_epoch (
int) – Run validation every N epochs.callbacks (
List[Callback] |None) – List of callbacks to use.logger (
Logger|List[Logger] |bool) – Logger(s) to use. True = CSVLogger, False = no logging.enable_progress_bar (
bool) – Whether to show progress bars.enable_checkpointing (
bool) – Recorded for API compatibility. The trainer saves no checkpoints by itself and creates no checkpoint directory; add aModelCheckpointcallback to save checkpoints.default_root_dir (
str|None) – Default root directory for logs and checkpoints.gradient_clip_val (
float|None) – Value for gradient clipping.gradient_clip_algorithm (
str) – Gradient clipping algorithm (‘norm’ or ‘value’).accumulate_grad_batches (
int) – Number of batches to accumulate gradients over before each optimizer step. Must be >= 1.devices (
int|List[int] |str) – Devices to use for training.'auto'uses all visible JAX devices; anintselects the first N; a list selects by index.Distributed training strategy. The active training loop drives a single optimizer on the selected device(s); multi-host strategies are provided but not exercised by
fit().Deprecated since version 0.4.0: Passing any value other than
'auto'(orNone) is deprecated and the argument will be removed in 1.0.0.precision (
str) –Requested compute precision (
'32','16','bf16', optionally with a'-mixed'/'-true'suffix). Recorded for API compatibility; the loop currently computes in float32 and emits a warning for any non-32 value. Cast your model/data explicitly for reduced precision.Deprecated since version 0.4.0: Values other than
'32'/'32-true'are deprecated and the argument will be removed in 1.0.0.deterministic (
bool) –Recorded for API compatibility. Determinism in JAX is governed by the PRNG keys you use and XLA flags; this flag does not itself change execution.
Deprecated since version 0.4.0: Passing
Trueis deprecated and the argument will be removed in 1.0.0. Useseedinstead.benchmark (
bool) –Recorded for API compatibility; has no effect (XLA autotunes and there is no cuDNN benchmark switch in JAX). A warning is emitted if set.
Deprecated since version 0.4.0: Passing
Trueis deprecated and the argument will be removed in 1.0.0.seed (
int|None) – Random seed for reproducibility. Seeds both NumPy andbrainstate.
- Raises:
ValueError – If
gradient_clip_algorithm,accumulate_grad_batches,precisionorstrategyis invalid.- Warns:
BraintoolsDeprecationWarning – If
precision,deterministic,benchmarkorstrategyis given a non-default value.
Notes
Only the first optimizer returned by
LightningModule.configure_optimizers()is stepped by the training loop; configuring several emits a warning. Freeze parameters withLightningModule.freeze()before callingfit()to exclude them from gradients and optimizer updates.Examples
Fit a linear regressor, then evaluate it and predict with it:
>>> import numpy as np >>> import jax.numpy as jnp >>> import brainstate >>> import braintools >>> class Regressor(braintools.trainer.LightningModule): ... def __init__(self): ... super().__init__() ... self.linear = brainstate.nn.Linear(2, 1) ... def __call__(self, x): ... return self.linear(x) ... def training_step(self, batch, batch_idx): ... x, y = batch ... loss = jnp.mean((self(x) - y) ** 2) ... self.log('mse', loss, prog_bar=True) ... return {'loss': loss} ... def validation_step(self, batch, batch_idx): ... x, y = batch ... return {'loss': jnp.mean((self(x) - y) ** 2)} ... def test_step(self, batch, batch_idx): ... x, y = batch ... return {'loss': jnp.mean((self(x) - y) ** 2)} ... def predict_step(self, batch, batch_idx): ... x, _ = batch ... return self(x) ... def configure_optimizers(self): ... return braintools.optim.Adam(lr=0.1) >>> rng = np.random.default_rng(0) >>> X = rng.normal(size=(64, 2)).astype(np.float32) >>> y = X @ np.array([[1.0], [-2.0]], dtype=np.float32) + 0.5 >>> train_loader = braintools.trainer.DataLoader((X, y), batch_size=16, shuffle=True, seed=0) >>> val_loader = braintools.trainer.DataLoader((X, y), batch_size=32) >>> brainstate.random.seed(0) >>> model = Regressor() >>> trainer = braintools.trainer.Trainer( ... max_epochs=30, logger=False, enable_progress_bar=False, ... enable_checkpointing=False) >>> trainer.fit(model, train_loader, val_loader) >>> trainer.current_epoch, trainer.global_step (29, 120) >>> trainer.callback_metrics['val_loss'] < 1e-3 True >>> trainer.test(dataloaders=val_loader, verbose=False)['test_loss'] < 1e-3 True >>> predictions = trainer.predict(dataloaders=val_loader) >>> len(predictions), predictions[0].shape (2, (32, 1))
With callbacks and CSV logging under a run directory:
>>> import os, shutil, tempfile >>> tmp = tempfile.mkdtemp() >>> brainstate.random.seed(0) >>> model = Regressor() >>> trainer = braintools.trainer.Trainer( ... max_epochs=100, ... callbacks=[ ... braintools.trainer.ModelCheckpoint( ... dirpath=os.path.join(tmp, 'checkpoints'), monitor='val_loss'), ... braintools.trainer.EarlyStopping( ... monitor='val_loss', patience=5, min_delta=1e-4), ... ], ... logger=braintools.trainer.CSVLogger(tmp, name='run', version='v0'), ... enable_progress_bar=False, ... default_root_dir=tmp, ... ) >>> trainer.fit(model, train_loader, val_loader) >>> trainer.current_epoch < 99 # early stopping ended the run True >>> sorted(os.listdir(os.path.join(tmp, 'run', 'v0'))) ['hparams.yaml', 'metrics.csv'] >>> 'last.ckpt' in os.listdir(os.path.join(tmp, 'checkpoints')) True >>> shutil.rmtree(tmp)
- fit(model, train_dataloaders=None, val_dataloaders=None, ckpt_path=None)[source]#
Run the full training loop.
- Parameters:
model (
LightningModule) – Model to train.train_dataloaders (
DataLoader|None) – Training data loader.val_dataloaders (
DataLoader|List[DataLoader] |None) – Validation data loader(s).ckpt_path (
str|None) – Path to a checkpoint to resume from. Restored are the model weights, the optimizer state, the epoch and global-step counters, and the state of the LR schedulers returned byconfigure_optimizers(the ones the trainer steps once per epoch). Training continues at epochcheckpoint_epoch + 1up tomax_epochs(the total, not additional, number of epochs), with the schedulers advanced to that epoch, so the LR matches an uninterrupted run. A checkpoint saved mid-epoch also resumes at the next epoch. A checkpoint without scheduler state (olderModelCheckpointfiles,CheckpointManageror hand-written ones) steps the fresh schedulers once per completed epoch instead and warns. Callback state is not restored. If the checkpoint already reached the last epoch, ormax_stepsoptimizer steps, a warning is emitted and nothing is trained.
Examples
>>> import numpy as np >>> import jax.numpy as jnp >>> import brainstate >>> import braintools >>> class Regressor(braintools.trainer.LightningModule): ... def __init__(self): ... super().__init__() ... self.linear = brainstate.nn.Linear(2, 1) ... def training_step(self, batch, batch_idx): ... x, y = batch ... return {'loss': jnp.mean((self.linear(x) - y) ** 2)} ... def validation_step(self, batch, batch_idx): ... x, y = batch ... return {'loss': jnp.mean((self.linear(x) - y) ** 2)} ... def configure_optimizers(self): ... return braintools.optim.Adam(lr=0.1) >>> rng = np.random.default_rng(0) >>> X = rng.normal(size=(32, 2)).astype(np.float32) >>> y = X.sum(axis=1, keepdims=True) >>> loader = braintools.trainer.DataLoader((X, y), batch_size=16) >>> brainstate.random.seed(0) >>> trainer = braintools.trainer.Trainer( ... max_epochs=3, logger=False, enable_progress_bar=False, ... enable_checkpointing=False) >>> trainer.fit(Regressor(), loader, val_dataloaders=loader) >>> trainer.global_step 6 >>> sorted(trainer.callback_metrics) ['train_loss', 'val_loss']
- predict(model=None, dataloaders=None, ckpt_path=None)[source]#
Run prediction.
- Parameters:
model (
LightningModule|None) – Model to use for prediction.dataloaders (
DataLoader|List[DataLoader] |None) – Prediction data loader(s).ckpt_path (
str|None) – Path to a checkpoint whose model weights are loaded first. The trainer’s epoch and step counters are left unchanged.
- Returns:
Predictions for each batch.
- Return type:
- test(model=None, dataloaders=None, ckpt_path=None, verbose=True)[source]#
Run testing.
- Parameters:
model (
LightningModule|None) – Model to test.dataloaders (
DataLoader|List[DataLoader] |None) – Test data loader(s).ckpt_path (
str|None) – Path to a checkpoint whose model weights are loaded first. The trainer’s epoch and step counters are left unchanged.verbose (
bool) – Whether to print results.
- Returns:
Test metrics.
- Return type:
- validate(model=None, dataloaders=None, ckpt_path=None, verbose=True)[source]#
Run validation.
- Parameters:
model (
LightningModule|None) – Model to validate. Uses self.model if not provided.dataloaders (
DataLoader|List[DataLoader] |None) – Validation data loader(s).ckpt_path (
str|None) – Path to a checkpoint whose model weights are loaded first. The trainer’s epoch and step counters are left unchanged.verbose (
bool) – Whether to print results.
- Returns:
Validation metrics.
- Return type: