lumin.evaluation package

Submodules

lumin.evaluation.ams module

lumin.evaluation.ams.ams_scan_quick(df, wgt_factor=1, br=0, syst_unc_b=0, pred_name='pred', targ_name='gen_target', wgt_name='gen_weight')

Scan across a range of possible prediction thresholds in order to maximise the Approximate Median Significance (https://arxiv.org/abs/1007.1727). Note that whilst this method is quicker than ams_scan_slow(), it sufferes from float precison. Not recommended for final evaluation.

Parameters:

• df (DataFrame) – DataFrame containing prediction data

• wgt_factor (float) – factor to reweight signal and background weights

• br (float) – background offset bias

• syst_unc_b (float) – fractional systemtatic uncertainty on background

• pred_name (str) – column to use as predictions

• targ_name (str) – column to use as truth labels for signal and background

• wgt_name (str) – column to use as weights for signal and background events

Return type:

Tuple[float, float]

Returns:

maximum AMS prediction threshold corresponding to maximum AMS

lumin.evaluation.ams.ams_scan_slow(df, wgt_factor=1, br=0, syst_unc_b=0, use_stat_unc=False, start_cut=0.9, min_events=10, pred_name='pred', targ_name='gen_target', wgt_name='gen_weight', show_prog=True)

Scan across a range of possible prediction thresholds in order to maximise the Approximate Median Significance (https://arxiv.org/abs/1007.1727). Note that whilst this method is slower than ams_scan_quick(), it does not suffer as much from float precison. Additionally it allows one to account for statistical uncertainty in AMS calculation.

Parameters:

• df (DataFrame) – DataFrame containing prediction data

• wgt_factor (float) – factor to reweight signal and background weights

• br (float) – background offset bias

• syst_unc_b (float) – fractional systemtatic uncertainty on background

• use_stat_unc (bool) – whether to account for the statistical uncertainty on the background

• start_cut (float) – minimum prediction to consider; useful for speeding up scan

• min_events (int) – minimum number of background unscaled events required to pass threshold

• pred_name (str) – column to use as predictions

• targ_name (str) – column to use as truth labels for signal and background

• wgt_name (str) – column to use as weights for signal and background events

• show_prog (bool) – whether to display progress and ETA of scan

Return type:

Tuple[float, float]

Returns:

maximum AMS prediction threshold corresponding to maximum AMS

lumin.evaluation.ams.calc_ams(s, b, br=0, unc_b=0)

Compute Approximate Median Significance (https://arxiv.org/abs/1007.1727)

Parameters:

• s (float) – signal weight

• b (float) – background weight

• br (float) – background offset bias

• unc_b (float) – fractional systemtatic uncertainty on background

Return type:

float

Returns:

Approximate Median Significance if b > 0 else -1

lumin.evaluation.ams.calc_ams_torch(s, b, br=0, unc_b=0)

Compute Approximate Median Significance (https://arxiv.org/abs/1007.1727) using Tensor inputs

Parameters:

• s (Tensor) – signal weight

• b (Tensor) – background weight

• br (float) – background offset bias

• unc_b (float) – fractional systemtatic uncertainty on background

Return type:

Tensor

Returns:

Approximate Median Significance if b > 0 else 1e-18 * s

Module contents