Skip to content

Core Package

The top-level idmtools_calibra package contains the central orchestration classes for running calibrations.

CalibManager

Python
from idmtools_calibra.calib_manager import CalibManager

The top-level entry point. Manages the multi-iteration calibration loop: sampling parameters, commissioning simulations, analyzing results, and updating the algorithm state.

Constructor

Python
CalibManager(
    task,
    map_sample_to_model_input_fn,
    sites,
    next_point,
    platform=None,
    name='calib_test',
    sim_runs_per_param_set=1,
    max_iterations=5,
    plotters=None,
    map_replicates_callback=None,
    directory='.'
)
Parameter Type Default Description
task ITask — An idmtools task (e.g. PythonTask, EMODTask)
map_sample_to_model_input_fn callable — Callback (simulation, sample_row) -> dict that applies one parameter sample to the task
sites List[CalibSite] — One or more calibration sites providing reference data and analyzers
next_point NextPointAlgorithm — Sampling algorithm instance (e.g. OptimTool, IMIS)
platform IPlatform None idmtools platform; defaults to the current platform context
name str 'calib_test' Calibration name; also used as the output directory name
sim_runs_per_param_set int 1 Number of replicate simulations per parameter sample
max_iterations int 5 Maximum number of calibration iterations
plotters List[BasePlotter] None Diagnostic plotters run after each iteration
map_replicates_callback callable None Callback for sweeping the replicate seed (defaults to Run_Number)
directory str '.' Root directory under which the calibration folder is created

Key Methods

run_calibration(**kwargs)

Start or resume a calibration. All parameters are passed as keyword arguments.

Keyword Type Default Description
resume bool False Resume an existing calibration
iteration int None Iteration number to resume from
iter_step str None Phase to resume from: 'commission', 'analyze', 'plot', 'next_point'
loop bool True Continue to subsequent iterations after resuming
max_iterations int None Override the max iterations set in the constructor
backup bool False Back up Calibration.json before resuming
dry_run bool False Preview resume action without executing
directory str None Override the calibration directory

open_for_reading(calibration_directory) (classmethod)

Open an existing calibration directory for reading results without executing anything.

Python
cm = CalibManager.open_for_reading('my_calibration')

Example

Python
from idmtools_calibra.calib_manager import CalibManager
from idmtools_calibra.algorithms.optim_tool import OptimTool
from idmtools.entities.command_task import CommandTask

task = CommandTask(command='python model.py')

calib = CalibManager(
    task=task,
    map_sample_to_model_input_fn=my_map_fn,
    sites=[my_site],
    next_point=OptimTool(params=[...]),
    name='my_calibration',
    max_iterations=10,
)

calib.run_calibration()

# Resume from iteration 3, analyze phase:
calib.run_calibration(resume=True, iteration=3, iter_step='analyze')

CalibSite

Python
from idmtools_calibra.calib_site import CalibSite

Abstract base class for calibration sites. A site pairs reference (observed) data with the analyzers that compare model output against it.

Constructor

Python
CalibSite(name)
Parameter Type Description
name str Unique name for this calibration site

Abstract Methods to Implement

Method Returns Description
get_analyzers() List[IAnalyzer] Return analyzer instances for this site
get_setup_functions() List[callable] Return functions that configure the task for this site
get_reference_data(reference_type=None) DataFrame Return the reference/observed data

Example

Python
from idmtools_calibra.calib_site import CalibSite
import pandas as pd

class MySite(CalibSite):
    def __init__(self):
        super().__init__(name='my_site')

    def get_analyzers(self):
        return [MyAnalyzer(site=self)]

    def get_setup_functions(self):
        return []

    def get_reference_data(self, reference_type=None):
        return pd.read_csv('reference/data.csv')

RMSESiteSingleChannel

Python
from idmtools_calibra.rmse_site import RMSESiteSingleChannel

A ready-to-use CalibSite subclass for RMSE-based calibration of a single output channel. Reads reference data from a CSV and wires up an RMSEAnalyzer automatically.

Constructor

Python
RMSESiteSingleChannel(
    name,
    reference_sources={'just_one': 'reference/output.csv'}
)
Parameter Type Default Description
name str — Site name
reference_sources dict {'just_one': 'reference/output.csv'} Dict mapping a key to the path of the reference CSV file. The CSV must have an independent column (first) and a dependent column (second).

Example

Python
from idmtools_calibra.rmse_site import RMSESiteSingleChannel

site = RMSESiteSingleChannel(
    name='solar_site',
    reference_sources={'data': 'reference/output.csv'}
)

IterationState

Python
from idmtools_calibra.iteration_state import IterationState

Holds all state for a single calibration iteration: parameter samples, simulation IDs, analyzer results, and timestamps. Serialized to and deserialized from Calibration.json to support resuming.

Key Attributes

Attribute Type Description
iteration int Current iteration number
calibration_name str Name of the calibration
calibration_directory str Path to the calibration output directory
samples_for_this_iteration dict Parameter samples drawn for this iteration
experiment_id str ID of the commissioned experiment
results dict Per-site analyzer results
all_results DataFrame Combined results across all sites
summary_table DataFrame Summary statistics

ResumeManager

Python
from idmtools_calibra.utilities.resume_manager import ResumeManager

Manages resuming a calibration from a specific iteration and phase. Restores CalibManager state, validates the requested iteration and step, and re-enters the calibration loop at exactly the right point. Used internally by CalibManager.run_calibration(resume=True, ...).

Constructor

Python
ResumeManager(
    calib_manager,
    iteration=None,
    iter_step=None,
    max_iterations=None,
    loop=True,
    backup=False,
    dry_run=False
)
Parameter Type Default Description
calib_manager CalibManager — The calibration manager to resume
iteration int None Iteration to resume from; auto-detects the latest if None
iter_step str None Phase to resume from: 'commission', 'analyze', 'plot', 'next_point'; auto-detects if None
max_iterations int None Override max iterations; uses calib_manager.max_iterations if None
loop bool True Continue to subsequent iterations after resuming
backup bool False Back up Calibration.json before resuming
dry_run bool False Print the resume plan without executing

Key Method

resume()

Restores all state and re-enters the calibration loop at the resolved iteration and step. Prints a summary of the resume plan before executing.

Python
rm = ResumeManager(calib_manager=calib, iteration=2, iter_step='analyze')
rm.resume()

Utility Functions

set_parameter_sweep_callback

Python
from idmtools_calibra.calib_manager import set_parameter_sweep_callback

Convenience callback compatible with CalibManager.map_replicates_callback. Sets a named parameter on the simulation task, supporting both task.config.parameters (EMOD-style) and task.parameters (JSON-configured tasks).

Python
set_parameter_sweep_callback(simulation, param='Run_Number', value=42)

generate_ll_all

Python
from idmtools_calibra.utilities.ll_all_generator import generate_ll_all
generate_ll_all(calib_manager, iteration, ll_all_name)

Method rebuild ll_all.csv.