lumin.nn.training package

Submodules

lumin.nn.training.fold_train module

lumin.nn.training.fold_train.fold_train_ensemble(fy, n_models, bs, model_builder, callback_partials=None, eval_metrics=None, train_on_weights=True, eval_on_weights=True, patience=10, max_epochs=200, shuffle_fold=True, shuffle_folds=True, bulk_move=True, live_fdbk=True, live_fdbk_first_only=True, live_fdbk_extra=True, live_fdbk_extra_first_only=False, savepath=PosixPath('train_weights'), verbose=False, log_output=False, plot_settings=<lumin.plotting.plot_settings.PlotSettings object>)

Main training method for Model. Trains a specified numer of models created by a ModelBuilder on data provided by a FoldYielder, and save them to savepath. Note, this does not return trained models, instead they are saved and must be loaded later. Instead this method returns results of model training. Each Model is trained on N-1 folds, for a FoldYielder with N folds, and the remaining fold is used as validation data. Training folds are loaded iteratively, and model evaluation takes place after each fold use (a sub-epoch), rather than after ever use of all folds (epoch). Training continues until:

• All of the training folds are used max_epoch number of times;

• Or validation loss does not decrease for patience number of training folds; (or cycles, if using an AbsCyclicCallback);

• Or a callback triggers trainign to stop, e.g. OneCycle

Depending on the live_fdbk arguments, live plots of losses and other metrics may be shown during training, if running in Jupyter. By default, a live plot with extra information will be shown for training the first model, and afterwards no live plots will be shown. Shoing the live plot slightly slows down the training, but can help highlight problems without having to wait to the end. Thererfore this compromises between showing useful information and training speed, since any problems should hopefully be visible in the first model.

Once training is finished, the state with the lowest validation loss is loaded, evaluated, and saved.

Parameters

• fy (FoldYielder) – FoldYielder interfacing ot training data

• n_models (int) – number of models to train

• bs (int) – batch size. Number of data points per iteration

• model_builder (ModelBuilder) – ModelBuilder creating the networks to train

• callback_partials (Optional[List[partial]]) – optional list of functools.partial, each of which will a instantiate Callback when called

• eval_metrics (Optional[Dict[str, EvalMetric]]) – list of instantiated EvalMetric. At the end of training, validation data and model predictions will be passed to each, and the results printed and saved

• train_on_weights (bool) – If weights are present in training data, whether to pass them to the loss function during training

• eval_on_weights (bool) – If weights are present in validation data, whether to pass them to the loss function during validation

• patience (int) – number of folds (sub-epochs) or cycles to train without decrease in validation loss before ending training (early stopping)

• max_epochs (int) – maximum number of epochs for which to train

• live_fdbk (bool) – whether or not to show any live feedback at all during training (slightly slows down training, but helps spot problems)

• live_fdbk_first_only (bool) – whether to only show live feedback for the first model trained (trade off between time and problem spotting)

• live_fdbk_extra (bool) – whether to show extra information live feedback (further slows training)

• live_fdbk_extra_first_only (bool) – whether to only show extra live feedback information for the first model trained (trade off between time and information)

• shuffle_fold (bool) – whether to tell BatchYielder to shuffle data

• shuffle_folds (bool) – whether to shuffle the order of the training folds

• bulk_move (bool) – whether to pass all training data to device at once, or by minibatch. Bulk moving will be quicker, but may not fit in memory.

• savepath (Path) – path to to which to save model weights and results

• verbose (bool) – whether to print out extra information during training

• log_output (bool) – whether to save printed results to a log file rather than printing them

• plot_settings (PlotSettings) – PlotSettings class to control figure appearance

Return type

Tuple[List[Dict[str, float]], List[Dict[str, List[float]]], List[Dict[str, float]]]

Returns

• results list of validation losses and other eval_metrics results, ordered by model training. Can be used to create an Ensemble.

• histories list of loss histories, ordered by model training

• cycle_losses if an AbsCyclicCallback was passed, list of validation losses at the end of each cycle, ordered by model training. Can be passed to Ensemble.

lumin.nn.training.metric_logger module

class lumin.nn.training.metric_logger.MetricLogger(loss_names, n_folds, autolog_scale=True, extra_detail=True, plot_settings=<lumin.plotting.plot_settings.PlotSettings object>)

Bases: object

Provides live feedback during training showing a variety of metrics to help highlight problems or test hyper-parameters without completing a full training.

Parameters

• loss_names (List[str]) – List of names of losses which will be passed to the logger in the order in which they will be passed. By convention the first name will be used as the training loss when computing the ratio of training to validation losses

• n_folds (int) – Number of folds present in the training data. The logger assumes that one of these folds is for validation, and so 1 training epoch = (n_fold-1) folds.

• autolog_scale (bool) – Whether to automatically change the scale of the y-axis for loss to logarithmic when the current loss drops below one 50th of its starting value

• extra_detail (bool) – Whether to include extra detail plots (loss velocity and training validation ratio), slight slower but potentially useful.

• plot_settings (PlotSettings) – PlotSettings class to control figure appearance

Examples::

>>> metric_log = MetricLogger(loss_names=['Train', 'Validation'], n_folds=train_fy.n_folds)
>>> val_losses = []
>>> metric_log.reset()  # Initialises plots and variables
>>> for epoch in epochs:
>>>     for fold in train_folds:
>>>         # train for one fold (subepoch)
>>>         metric_log.update_vals([train_loss, val_loss], best=best_val_loss)
>>>     metric_log.update_plot()
>>> plt.clf()

add_loss_name(name)

Adds an additional loss name to the loss names displayed. The associated losses will be set to zero for any prior subepochs which have elapsed already.

Parameters

name (str) – name of loss to be added

Return type

None

reset()

Resets/initialises the logger’s values and plots, and produces a placeholder plot. Should be called prior to update_vals or update_plot.

Return type

None

update_plot(best=None)

Updates the plot(s), Optionally showing the user-chose best loss achieved.

Parameters

best (Optional[float]) – the value of the best loss achieved so far

Return type

None

update_vals(vals)

Appends values to the losses. This is interpreted as one subepoch having elapsed (i.e. one training fold).

Parameters

vals (List[float]) – loss values from the last subepoch in the order of loss_names

Return type

None

Module contents