Calibration#

class Calibration(pars, calib_pars=None, weights=None, verbose=True, keep_db=False, **kwargs)[source]#

Bases: prettyobj

A class to handle calibration of FPsim objects. Uses the Optuna hyperparameter optimization library (optuna.org).

Note: running a calibration does not guarantee a good fit! You must ensure that you run for a sufficient number of iterations, have enough free parameters, and that the parameters have wide enough bounds. Please see the tutorial on calibration for more information.

Parameters:

  • sim (Sim) – the simulation to calibrate

  • calib_pars (dict) – a dictionary of the parameters to calibrate of the format dict(key1=[best, low, high])

  • weights (dict) – a custom dictionary of weights for each output

  • n_trials (int) – the number of trials per worker

  • n_workers (int) – the number of parallel workers (default: maximum)

  • total_trials (int) – if n_trials is not supplied, calculate by dividing this number by n_workers

  • name (str) – the name of the database (default: ‘fpsim_calibration’)

  • db_name (str) – the name of the database file (default: ‘fpsim_calibration.db’)

  • keep_db (bool) – whether to keep the database after calibration (default: false)

  • storage (str) – the location of the database (default: sqlite)

  • label (str) – a label for this calibration object

  • verbose (bool) – whether to print details of the calibration

  • kwargs (dict) – passed to cv.Calibration()

Returns:

A Calibration object

Methods

set_optuna_defaults()[source]#

Create a (mutable) dictionary with default global settings

configure_optuna(**kwargs)[source]#

Update Optuna configuration, if required

validate_pars()[source]#

Ensure parameters are in the correct format. Two formats are permitted: either a dict of arrays or lists in order best-low-high, e.g.:

calib_pars = dict(
    exposure_factor           = [1.0, 0.5,  1.5],
    maternal_mortality_factor = [1,   0.75, 3.0],
)

Or the same thing, as a dict of dicts:

calib_pars = dict(
    exposure_factor           = dict(best=1.0, low=0.5,  high=1.5),
    maternal_mortality_factor = dict(best=1,   low=0.75, high=3.0),
)
run_exp(pars, return_exp=False, **kwargs)[source]#

Create and run an experiment

run_trial(trial)[source]#

Define the objective for Optuna

worker()[source]#

Run a single worker

run_workers()[source]#

Run multiple workers in parallel

remove_db()[source]#

Remove the database file if keep_db is false and the path exists.

make_study()[source]#

Make a study, deleting one if it already exists

calibrate(calib_pars=None, weights=None, verbose=None, **kwargs)[source]#

Actually perform calibration

parse_study()[source]#

Parse the study into a data frame

to_json(filename=None)[source]#

Convert the data to JSON

plot_trend(best_thresh=2)[source]#

Plot the trend in best mismatch over time

plot_all()[source]#

Plot every point: warning, very slow!

plot_best(best_thresh=2)[source]#

Plot only the points with lowest mismatch

plot_stride(npts=200)[source]#

Plot a fixed number of points in order across the results