fpsim.calibration module

Define the Calibration class

class fpsim.calibration.Calibration(pars, calib_pars=None, weights=None, verbose=True, keep_db=False, **kwargs)

Bases: sciris.sc_utils.prettyobj

A class to handle calibration of FPsim objects. Uses the Optuna hyperparameter optimization library (optuna.org).

Note: running a calibration does not guarantee a good fit! You must ensure that you run for a sufficient number of iterations, have enough free parameters, and that the parameters have wide enough bounds. Please see the tutorial on calibration for more information.

Parameters

  • sim (Sim) – the simulation to calibrate

  • calib_pars (dict) – a dictionary of the parameters to calibrate of the format dict(key1=[best, low, high])

  • weights (dict) – a custom dictionary of weights for each output

  • n_trials (int) – the number of trials per worker

  • n_workers (int) – the number of parallel workers (default: maximum)

  • total_trials (int) – if n_trials is not supplied, calculate by dividing this number by n_workers

  • name (str) – the name of the database (default: ‘fpsim_calibration’)

  • db_name (str) – the name of the database file (default: ‘fpsim_calibration.db’)

  • keep_db (bool) – whether to keep the database after calibration (default: false)

  • storage (str) – the location of the database (default: sqlite)

  • label (str) – a label for this calibration object

  • verbose (bool) – whether to print details of the calibration

  • kwargs (dict) – passed to cv.Calibration()

Returns

A Calibration object

set_optuna_defaults()

Create a (mutable) dictionary with default global settings

configure_optuna(**kwargs)

Update Optuna configuration, if required

validate_pars()

Ensure parameters are in the correct format. Two formats are permitted: either a dict of arrays or lists in order best-low-high, e.g.:

calib_pars = dict(
    exposure_factor           = [1.0, 0.5,  1.5],
    maternal_mortality_factor = [1,   0.75, 3.0],
)

Or the same thing, as a dict of dicts:

calib_pars = dict(
    exposure_factor           = dict(best=1.0, low=0.5,  high=1.5),
    maternal_mortality_factor = dict(best=1,   low=0.75, high=3.0),
)
run_exp(pars, return_exp=False, **kwargs)

Create and run an experiment

run_trial(trial)

Define the objective for Optuna

worker()

Run a single worker

run_workers()

Run multiple workers in parallel

remove_db()

Remove the database file if keep_db is false and the path exists.

make_study()

Make a study, deleting one if it already exists

calibrate(calib_pars=None, weights=None, verbose=None, **kwargs)

Actually perform calibration

summarize()
parse_study()

Parse the study into a data frame

to_json(filename=None)

Convert the data to JSON

plot_trend(best_thresh=2)

Plot the trend in best mismatch over time

plot_all()

Plot every point: warning, very slow!

plot_best(best_thresh=2)

Plot only the points with lowest mismatch

plot_stride(npts=200)

Plot a fixed number of points in order across the results