fpsim.calibration module

Define the Calibration class

class fpsim.calibration.Calibration(pars, calib_pars=None, weights=None, verbose=True, keep_db=False, **kwargs)

Bases: sciris.sc_utils.prettyobj

A class to handle calibration of FPsim objects. Uses the Optuna hyperparameter optimization library (optuna.org).

Note: running a calibration does not guarantee a good fit! You must ensure that you run for a sufficient number of iterations, have enough free parameters, and that the parameters have wide enough bounds. Please see the tutorial on calibration for more information.

  • sim (Sim) – the simulation to calibrate

  • calib_pars (dict) – a dictionary of the parameters to calibrate of the format dict(key1=[best, low, high])

  • weights (dict) – a custom dictionary of weights for each output

  • n_trials (int) – the number of trials per worker

  • n_workers (int) – the number of parallel workers (default: maximum)

  • total_trials (int) – if n_trials is not supplied, calculate by dividing this number by n_workers

  • name (str) – the name of the database (default: ‘fpsim_calibration’)

  • db_name (str) – the name of the database file (default: ‘fpsim_calibration.db’)

  • keep_db (bool) – whether to keep the database after calibration (default: false)

  • storage (str) – the location of the database (default: sqlite)

  • label (str) – a label for this calibration object

  • verbose (bool) – whether to print details of the calibration

  • kwargs (dict) – passed to cv.Calibration()


A Calibration object


Create a (mutable) dictionary with default global settings


Update Optuna configuration, if required


Ensure parameters are in the correct format. Two formats are permitted: either a dict of arrays or lists in order best-low-high, e.g.:

calib_pars = dict(
    exposure_factor           = [1.0, 0.5,  1.5],
    maternal_mortality_factor = [1,   0.75, 3.0],

Or the same thing, as a dict of dicts:

calib_pars = dict(
    exposure_factor           = dict(best=1.0, low=0.5,  high=1.5),
    maternal_mortality_factor = dict(best=1,   low=0.75, high=3.0),
run_exp(pars, return_exp=False, **kwargs)

Create and run an experiment


Define the objective for Optuna


Run a single worker


Run multiple workers in parallel


Remove the database file if keep_db is false and the path exists.


Make a study, deleting one if it already exists

calibrate(calib_pars=None, weights=None, verbose=None, **kwargs)

Actually perform calibration


Parse the study into a data frame


Convert the data to JSON


Plot the trend in best mismatch over time


Plot every point: warning, very slow!


Plot only the points with lowest mismatch


Plot a fixed number of points in order across the results