Welcome to idmtools_calibra

idmtools_calibra is a collection of Python scripts and utilities to aid in model calibration. The idmtools package should be used for streamlining the process of running disease modeling simulations. Additional information can be found at Welcome to idmtools.

See Welcome to idmtools for a diagram showing how idmtools and each of the related packages are used in an end-to-end workflow using EMOD as the disease transmission model.

idmtools_calibra installation

Follow the steps below to install idmtools_calibra.

Prerequisites

First, ensure the following prerequisites are met.

• Windows 10 Pro or Enterprise, Linux, or Mac

• Python 3.6.2 or 3.7 64-bit (https://www.python.org/downloads/release)

• A file that indicates the pip index-url:

  • Windows

  • Linux

  In C:\Users\Username\pip\pip.ini, containing the following:

  [global]
  index-url = https://packages.idmod.org/api/pypi/pypi-production/simple
  

  In $HOME/.config/pip/pip.conf, containing the following:

  [global]
  index-url = https://packages.idmod.org/api/pypi/pypi-production/simple
  

Installation instructions

1. Open a command prompt and create a virtual environment in any directory you choose. The command below names the environment “v-calibra”, but you may use any desired name:

   python -m venv v-calibra
   

2. Activate the virtual environment:

   • Windows

   • Linux

   Enter the following:

   v-calibra\Scripts\activate
   

   Enter the following:

   source v-calibra/bin/activate
   

3. Install idmtools_calibra packages:

   pip install idmtools_calibra
   

   If you are on Python 3.6, also run:

   pip install dataclasses
   

   If you are on Linux, also run:

   pip install keyrings.alt
   

4. When you are finished, deactivate the virtual environment by entering the following at a command prompt:

   deactivate
   

idmtools_calibra overview

Add overview of library here

idmtools_calibra

idmtools_calibra package

Subpackages

idmtools_calibra.algorithms package

Subpackages

idmtools_calibra.algorithms.pbnb package

Submodules

idmtools_calibra.algorithms.pbnb.c_sub_region module

Created on Fri Jun 23 14:21:27 2017

@author: TingYu Ho

class idmtools_calibra.algorithms.pbnb.c_sub_region.cSubRegion(coordinate_lower, coordinate_upper, params)

Bases: object

idmtools_calibra.algorithms.pbnb.fun_pbnb_support_functions module

idmtools_calibra.algorithms.pbnb.fun_pbnb_support_functions.fun_sample_points_generator_deterministic(l_subr, i_n_sampling, i_n_rep, s_stage, l_para)

idmtools_calibra.algorithms.pbnb.fun_pbnb_support_functions.fun_sample_points_generator_noise(l_subr, i_n_sampling, i_n_rep, s_stage, l_para)

idmtools_calibra.algorithms.pbnb.fun_pbnb_support_functions.turn_to_power(list, power)

idmtools_calibra.algorithms.pbnb.fun_pbnb_support_functions.fun_results_organizer_deterministic(l_subr, df_testing_samples, params)

idmtools_calibra.algorithms.pbnb.fun_pbnb_support_functions.fun_results_organizer_noise(l_subr, df_testing_samples, i_n_k, i_n_elite_worst, s_stage, params)

idmtools_calibra.algorithms.pbnb.fun_pbnb_support_functions.fun_order_subregion(c_subr)

idmtools_calibra.algorithms.pbnb.fun_pbnb_support_functions.fun_replication_update(l_subr, i_n_rep, f_alpha)

idmtools_calibra.algorithms.pbnb.fun_pbnb_support_functions.fun_order_region(l_subr)

idmtools_calibra.algorithms.pbnb.fun_pbnb_support_functions.fun_pruning_indicator(l_subr, f_CI_u)

idmtools_calibra.algorithms.pbnb.fun_pbnb_support_functions.fun_maintaining_indicator(l_subr, f_CI_l)

idmtools_calibra.algorithms.pbnb.fun_pbnb_support_functions.fun_elite_indicator(l_subr, f_CI_l)

idmtools_calibra.algorithms.pbnb.fun_pbnb_support_functions.fun_worst_indicator(l_subr, f_CI_u)

idmtools_calibra.algorithms.pbnb.fun_pbnb_support_functions.fun_quantile_update(l_subr, f_delta)

idmtools_calibra.algorithms.pbnb.fun_pbnb_support_functions.fun_ci_builder(l_subr, pd_order_z, f_delta_k, f_alpha_k, f_epsilon)

idmtools_calibra.algorithms.pbnb.fun_pbnb_support_functions.fun_pruning_labeler(l_subr)

idmtools_calibra.algorithms.pbnb.fun_pbnb_support_functions.fun_maintaining_labeler(l_subr)

idmtools_calibra.algorithms.pbnb.fun_pbnb_support_functions.fun_reg_branching(c_subr, i_n_branching, params, s_branching_dim)

idmtools_calibra.algorithms.pbnb.fun_pbnb_support_functions.fun_plot2D(l_subr, l_initial_coordinate_lower, l_initial_coordinate_upper, params, str_k, s_running_file_name, i_iteration)

idmtools_calibra.algorithms.pbnb.m_intial_paramters_setting module

Created on Fri Jun 23 22:15:00 2017

@author: TingYu Ho

idmtools_calibra.algorithms.pbnb.optim_tool_pbnb module

class idmtools_calibra.algorithms.pbnb.optim_tool_pbnb.OptimToolPBnB(params, s_running_file_name, s_problem_type, constrain_sample_fn=<function OptimToolPBnB.<lambda>>, f_delta=0.6, f_alpha=0.5, i_k_b=3, i_n_branching=4, i_c=120, i_replication=1, i_stopping_max_k=10, i_max_num_simulation_per_run=1500, f_elite_worst_sampling_para=4e-05)

Bases: NextPointAlgorithm

get_samples_for_iteration(iteration)

fun_generate_samples_from_df(df_samples)

fun_probability_branching_and_bound(iteration)

logging_saver(iteration)

print_results_for_iteration()

set_results_for_iteration(iteration, results)

get_state()

set_state(state, iteration)

end_condition()

get_results_to_cache(results)

get_final_samples()

update_summary_table(iteration_state, previous_results)

copy from NExtPointAlgorithm, output: all_results, summary_table

get_param_names()

cleanup()

Submodules

idmtools_calibra.algorithms.fisher_inf_matrix module

idmtools_calibra.algorithms.fisher_inf_matrix.perturbed_points(center, xmin, xmax, m=10, n=5, sample_size=1, resolution_raw=None)

Having the estimated optimal point (θ*), the next step is generating a set of perturbed points (samples). At this step, you can specify the size of perturbation (resolution). In case you do not input the perturbation size (resolution_raw=None), the script will pick a perturbation using the given range for each parameter (Xmin-Xmax). You need to allocate per-realization (M) and cross-realization (N). Note that \(M>p^2\), where p is the number of parameters (size of θ). The number of cross-realization (N) should be set depending on the uncertainty of the model at the given final point (θ*). To choose N, you can use Chebyshev’s inequality.

You can allocate number of replicates of the model (n). The log-likelihood at each point obtains from n replicates of the model with different run numbers. Then these n replicates are used to compute the likelihood at that point. When we have a model with high uncertainty, the easiest way to compute the likelihood might be taking average of the multiple (n) replicates of the model run to compute the likelihood. Note that the algorithm only accepts n=1 now. But the Cramer Rao script has the potential to accept higher n, whenever a smoothing technique which requires multiple replicates of the model for computing the likelihood be added to the Analyzers.

Parameters:

• center – center point 1xp nparray

• xmin – minimum of parameters 1xp nparray

• xmax – maximum of parameters 1xp nparray

• m – number of Hessian estimates scalar-positive integer

• n – number of pseudodata vectors scalar-positive integer

• sample_size – sample size scalar-positive integer

• resolution_raw – minimum meaningful perturbation for each parameter 1xp nparray

Returns:

perturbed points (4MNn x 4+p) nparray

Return type:

X_perturbed

idmtools_calibra.algorithms.fisher_inf_matrix.compute_fisher_inf_matrix(center_point, df_ll_points, data_columns)

Compute the Fisher Information matrix using the LL of perturbed points

Computation of the Fisher information matrix (covariance matrix) This step returns a p×p covariance matrix, called Σ.

Atiye Alaeddini, 12/15/2017

Parameters:

• center_point – center point (1 x p) nparray

• df_ll_points – Log Likelihood of points DataFrame

• data_columns – List of columns to filter points with

Returns:

Fisher Information matrix (p x p) np array

Return type:

Fisher

idmtools_calibra.algorithms.fisher_inf_matrix.sample_cov_ellipse(cov, pos, num_of_pts=10)

Sample ‘num_of_pts’ points from the specified covariance matrix (cov).

Parameters:

• cov – 2-D array_like, The covariance matrix (inverse of fisher matrix). It must be symmetric and positive-semidefinite for proper sampling.

• pos – 1-D array_like, The location of the center of the ellipse, Mean of the multi variate distribution

• num_of_pts – The number of sample points.

Returns:

ndarray of the drawn samples, of shape (num_of_pts,).

idmtools_calibra.algorithms.fisher_inf_matrix.trunc_gauss(mu, sigma, low_bound, high_bound, num_of_pts, batch_size=100)

idmtools_calibra.algorithms.fisher_inf_matrix.div0(a, b)

ignore / 0, div0( [-1, 0, 1], 0 ) -> [0, 0, 0]

idmtools_calibra.algorithms.fisher_inf_matrix.near_pd(a, nit=10)

idmtools_calibra.algorithms.generic_iterative_next_point module

class idmtools_calibra.algorithms.generic_iterative_next_point.GenericIterativeNextPoint(initial_state)

Bases: NextPointAlgorithm

Represents a Generic Next Point allowing thew Calibtool to function as a more generic iterative process. Here a dictionary needs to be passed as the state. For example:

initial_state = [{
 'Run_Number': rn
 } for rn in range(2)]

Then the results of the analyzers are stored in the self.data associating iteration with results. Both the initial state and the results are stored there allowing to easily refer to it:

self.data = [
                {
                'Run_Number': 1
                'results':{
                    'what_comes_from_analyzers':{}
                },
                ...
            ]

Note that the results needs to be contained in a Dictionary. If you want to leverage pandas.DataFrame instead, you should use OptimTool.

set_state(state, iteration)

cleanup()

update_iteration(iteration)

Update the current iteration state of the algorithm.

get_param_names()

get_samples_for_iteration(iteration)

get_state()

prep_for_dict(df)

Utility function allowing to transform a DataFrame into a dict removing null values

set_results_for_iteration(iteration, results)

end_condition()

get_final_samples()

update_summary_table(iteration_state, previous_results)

Returns a summary table of the form:

[result1 result2 results_total param1 param2 iteration simIds] index = sample Used by OptimTool and IMIS algorithm

get_results_to_cache(results)

idmtools_calibra.algorithms.gpc module

class idmtools_calibra.algorithms.gpc.GPC(x_cols, y_col, training_data, param_info, kernel_mode='RBF', kernel_params=None, verbose=False, debug=False, **kwargs)

Bases: object

classmethod from_config(config_fn)

classmethod from_dict(config)

set_training_data(new_training_data)

save(save_to=None)

define_kernel(params)

kernel_xx(x, theta)

static kernel_xp(x, p, theta)

kxx_gpu_wrapper(x, theta, deriv=-1)

kxp_gpu_wrapper(x, p, theta)

assign_rep(sample)

expectation_propagation(theta)

find_posterior_mode(theta, f_guess=None, tol_grad=1e-06, max_iter=100)

negative_log_marginal_likelihood(theta)

negative_log_marginal_likelihood_and_gradient(theta, f_guess=None)

static func_wrapper(f, cache_size=100)

laplace_predict(theta, f_hat, p)

ep_predict(theta, p)

optimize_hyperparameters(x0, bounds=(), k=-1, eps=0.01, disp=True, maxiter=15000)

evaluate(data)

plot_data(samples_to_circle=None)

plot_histogram()

plot(x_center, res=10)

plot_errors(train, test, mean_col, var_predictive_col, truth_col=None, figsize=(16, 10))

idmtools_calibra.algorithms.imis module

class idmtools_calibra.algorithms.imis.IMIS(Incremental Mixture Importance Sampling)

Bases: NextPointAlgorithm

Algorithm ported from R code: http://cran.r-project.org/web/packages/IMIS

Full description from Adrian Raftery and Le Bao (2009): http://www.stat.washington.edu/research/reports/2009/tr560.pdf

The basic idea of IMIS is that points with high importance weights are in areas where the target density is underrepresented by the importance sampling distribution. At each iteration, a multivariate normal distribution centered at the point with the highest importance weight is added to the current importance sampling distribution, which thus becomes a mixture of such functions and of the prior. In this way underrepresented parts of the parameter space are successively identified and are given representation, ending up with an iteratively constructed importance sampling distribution that covers the target distribution well.

validate_parameters()

Ensure valid parameter ranges: ‘samples_per_iteration’ is used to select N-closest points to maximum-weight sample, so it can’t exceed ‘n_initial_samples’. It is also used to estimate weighted covariance, so it cannot be smaller than the dimensionality of the samples.

choose_initial_samples()

set_initial_samples()

Set the initial samples points for the algorithm. If initial_samples parameter is an array, use those values as initial samples. Otherwise, if initial_samples parameter is a number, draw the specified number randomly from the prior distribution.

add_samples(samples, iteration)

get_next_samples_for_iteration(iteration)

choose_next_point_samples(iteration)

generate_variables_from_data()

restore some properties from self.data

get_samples_for_iteration(iteration)

update_iteration(iteration)

Initial Stage (iteration: k = 0)

1. Sample N inputs \(\theta_1, \theta_2, ... , \theta_N\) from the prior distribution \(p(\theta)\).

2. For each \(\theta_i\), calculate the likelihood \(L_i\), and form the importance weights: \(w^{(0)}_i = L_i / sum(L_j)\).

Importance Sampling Stage (iteration: k > 0; samples_per_iteration: B)

3. Calculate the likelihood of the new inputs and combine the new inputs with the previous ones. Form the importance weights: \(w^{(k)_i} = c * L_i * p(\theta_i) / q^{(k)}(\theta_i)\), where c is chosen so that the weights add to 1, \(q^{(k)}\) is the mixture sampling distribution: \(q^{(k)} = (N_0/N_k) * p + (B/N_k) * sum(H_s)\) where \(H_s\) is the Sth multivariate normal distribution, and \(N_k = N_0 + B_k\) is the total number of inputs up to iteration k.

update_state(iteration)

Update the next-point algorithm state and select next samples.

update_gaussian()

Importance Sampling Stage (iteration: k > 0; samples_per_iteration: B)

1. Choose the current maximum weight input as the center \(\theta^{(k)}\). Estimate \(\Sigma^{(k)}\) from the weighted covariance of the B inputs with the smallest Mahalanobis distances to \(\theta^{(k)}\), where the distances are calculated with respect to the covariance of the prior distribution and the weights are taken to be proportional to the average of the importance weights and \(1/N_k\).

2. Sample ‘samples_per_iteration’ new inputs from a multivariate Gaussian distribution \(H_k\) with covariance matrix \(\Sigma^{(k)}\).

update_gaussian_center()

Choose the current maximum weight input as the center point for the next iteration of multivariate-normal sampling.

weighted_distances_from_center()

Calculate the covariance-weighted distances from the current maximum-weight sample. N.B. Using normalized Euclid instead of Mahalanobis if we’re just going to diagonalize anyways.

update_gaussian_covariance(distances)

Calculate the covariance of the next-iteration of multivariate-normal sampling from the “samples_per_iteration” closest samples.

get_param_names()

verify_valid_samples(next_samples)

Resample from next-point function until all samples have non-zero prior.

next_point_fn()

IMIS next-point sampling from multivariate normal centered on the maximum weight.

update_gaussian_probabilities(iteration)

Calculate the probabilities of all sample points as estimated from the multivariate-normal probability distribution function centered on the maximum weight and with covariance fitted from the most recent iteration.

calculate_weighted_covariance(samples, weights, center)

A weighted covariance of sample points. N.B. The weights are normalized as in the R function “cov.wt”

end_condition()

Stopping Criterion:

The algorithm ends when the importance sampling weights are reasonably uniform. Specifically, we end the algorithm when the expected fraction of unique points in the resample is at least (1 - 1/e) = 0.632. This is the expected fraction when the importance sampling weights are all equal, which is the case when the importance sampling function is the same as the target distribution.

get_final_samples()

get_state()

set_state(state, iteration)

set_results_for_iteration(iteration, results)

cleanup()

restore(iteration_state)

idmtools_calibra.algorithms.next_point_algorithm module

class idmtools_calibra.algorithms.next_point_algorithm.NextPointAlgorithm

Bases: object

abstract set_state(state, iteration)

restore(iteration_state)

abstract cleanup()

abstract get_param_names()

abstract get_samples_for_iteration(iteration)

abstract get_state()

abstract set_results_for_iteration(iteration, results)

abstract end_condition()

abstract get_final_samples()

update_iteration(iteration)

Update the current iteration state of the algorithm.

prep_for_dict(df)

Utility function allowing to transform a DataFrame into a dict removing null values

static sample_from_function(function, N)

update_summary_table(iteration_state, previous_results)

Returns a summary table of the form:

[result1 result2 results_total param1 param2 iteration simIds] index = sample Used by OptimTool and IMIS algorithm

get_results_to_cache(results)

generate_samples_from_df(dfsamples)

idmtools_calibra.algorithms.optim_tool module

class idmtools_calibra.algorithms.optim_tool.OptimTool(params, constrain_sample_fn=<function OptimTool.<lambda>>, mu_r=0.1, sigma_r=0.02, center_repeats=2, samples_per_iteration=100.0, rsquared_thresh=0.5)

Bases: NextPointAlgorithm

The basic idea of OptimTool is

cleanup()

verify_param()

resolve_args(iteration)

add_samples(samples, iteration)

get_samples_for_iteration(iteration)

clamp(X)

set_results_for_iteration(iteration, results)

choose_initial_samples()

choose_samples_via_gradient_ascent(iteration)

choose_and_clamp_hypersphere_samples_for_iteration(iteration)

sample_hypersphere(N, state)

end_condition()

get_final_samples()

Resample Stage:

prep_for_dict(df)

Utility function allowing to transform a DataFrame into a dict removing null values

get_state()

set_state(state, iteration)

get_param_names()

static get_r(num_params, volume_fraction)

idmtools_calibra.algorithms.optim_tools_pspo module

class idmtools_calibra.algorithms.optim_tools_pspo.OptimToolPSPO(params, constrain_sample_fn, comps_per_iteration=10)

Bases: NextPointAlgorithm

OptimTool

The basic idea of OptimTool is

cleanup()

resolve_args(iteration)

add_samples(samples, iteration)

get_samples_for_iteration(iteration)

clamp(X)

set_results_for_iteration(iteration, results)

choose_initial_samples()

stochastic_newton_raphson(iteration)

choose_and_clamp_samples_for_iteration(iteration)

sample_simultaneous_perturbation(M, iteration, state)

end_condition()

get_final_samples()

prep_for_dict(df)

Utility function allowing to transform a DataFrame into a dict removing null values

get_state()

set_state(state, iteration)

get_param_names()

idmtools_calibra.algorithms.optim_tools_spsa module

class idmtools_calibra.algorithms.optim_tools_spsa.OptimToolSPSA(params, constrain_sample_fn, comps_per_iteration=10)

Bases: NextPointAlgorithm

The basic idea of OptimToolSPSA is

cleanup()

resolve_args(iteration)

add_samples(samples, iteration)

get_samples_for_iteration(iteration)

clamp(X)

set_results_for_iteration(iteration, results)

choose_initial_samples()

stochastic_newton_raphson(iteration)

choose_and_clamp_samples_for_iteration(iteration)

sample_simultaneous_perturbation(M, iteration, state, resolution=None)

end_condition()

get_final_samples()

Resample Stage:

prep_for_dict(df)

Utility function allowing to transform a DataFrame into a dict removing null values

get_state()

set_state(state, iteration)

get_param_names()

idmtools_calibra.algorithms.separatrix_bhm module

class idmtools_calibra.algorithms.separatrix_bhm.SeparatrixBHM(params, constrain_sample_fn=<function SeparatrixBHM.<lambda>>, implausibility_threshold=3, target_success_probability=0.7, num_past_iterations_to_include_in_metamodel=3, samples_per_iteration=32, samples_final_iteration=128, max_iterations=10, training_frac=0.8)

Bases: NextPointAlgorithm

Separatrix using Bayesian History Matching

The basic idea of Separatrix is that each simulation results in a success (+1) or a failure (-1), and the success probability varies as a function of the input parameters. We seek an isocline of the the latent success probability function.

cleanup()

resolve_args(iteration)

add_samples(samples, iteration)

get_samples_for_iteration(iteration)

set_results_for_iteration(iteration, results)

set_gpc_vec(iteration, gpc)

choose_initial_samples()

choose_samples_via_history_matching(iteration)

end_condition()

get_final_samples()

Return some number of samples from the residual non-implausible area:

prep_for_dict(df)

Utility function allowing to transform a DataFrame into a dict removing null values

get_state()

set_state(state, iteration)

get_param_names()

idmtools_calibra.analyzers package

Submodules

idmtools_calibra.analyzers.base_calibration_analyzer module

class idmtools_calibra.analyzers.base_calibration_analyzer.BaseCalibrationAnalyzer(uid=None, working_dir=None, parse=True, need_dir_map=False, filenames=None, reference_data=None, weight=1)

Bases: IAnalyzer

cache()

idmtools_calibra.analyzers.rmse_analyzer module

class idmtools_calibra.analyzers.rmse_analyzer.RMSEAnalyzer(site, dependent_column, independent_column, output_filename='output.csv')

Bases: BaseCalibrationAnalyzer

map(data, item)

In parallel for each simulation/work item, consume raw data from filenames and emit selected data.

Parameters:

• data – A dictionary associating filename with content for simulation data.

• item – IItem object that the passed data is associated with.

Returns:

Selected data for the given simulation/work item.

static set_custom_cost_fn(user_cost_fn)

rmse(df, data_column, reference_column)

compare(sample, data_column, reference_column)

reduce(all_data)

Combine the simulation data into a single table for all analyzed simulations.

idmtools_calibra.cli package

Submodules

idmtools_calibra.cli.commands module

idmtools_calibra.cli.utils module

idmtools_calibra.cli.utils.load_config_module(config_name)

idmtools_calibra.cli.utils.get_calib_manager(config_name, calib_callback='get_manager')

idmtools_calibra.cli.utils.read_calib_data(calib_path, force=False)

idmtools_calibra.output package

Submodules

idmtools_calibra.output.spatial_output module

class idmtools_calibra.output.spatial_output.SpatialOutput

Bases: object

classmethod from_bytes(bytes, filtered=False)

to_dict()

idmtools_calibra.plotters package

Submodules

idmtools_calibra.plotters.base_plotter module

class idmtools_calibra.plotters.base_plotter.BasePlotter(combine_sites=True)

Bases: object

get_iteration_directory()

get_plot_directory()

property all_results

abstract visualize(iteration_state)

static combine_by_site(site_name, analyzer_names, results)

Sum the result values over analyzers performed on the specified site and add SITE_total to results :param site_name: The name of the CalibSite :param analyzer_names: A list of CalibAnalyzer names corresponding to the site :param results: A pandas.DataFrame of results into which to add a combined site-analyzer-result column :return: None

idmtools_calibra.plotters.likelihood_plotter module

class idmtools_calibra.plotters.likelihood_plotter.LikelihoodPlotter(combine_sites=True)

Bases: BasePlotter

property param_names

property prior_fn

property directory

visualize(iteration_state)

plot_by_parameter_and_site()

plot_by_parameter(site='', **kwargs)

static plot1d_by_iteration(results, param, total, **kwargs)

idmtools_calibra.plotters.optim_tool_pbnb_plotter module

class idmtools_calibra.plotters.optim_tool_pbnb_plotter.OptimToolPBnBPlotter

Bases: BasePlotter

cleanup()

static plot_state_evolution(**kwargs)

visualize(iteration_state)

visualize_results()

idmtools_calibra.plotters.optim_tool_plotter module

class idmtools_calibra.plotters.optim_tool_plotter.OptimToolPlotter

Bases: BasePlotter

cleanup()

plot_state_evolution(**kwargs)

visualize(iteration_state)

visualize_results()

visualize_optimtool_diagnoistics()

cleanup_plot(calib_manager)

idmtools_calibra.plotters.optim_tool_spsa_plotter module

class idmtools_calibra.plotters.optim_tool_spsa_plotter.OptimToolSPSAPlotter

Bases: BasePlotter

cleanup()

plot_state_evolution(**kwargs)

visualize(iteration_state)

visualize_results()

visualize_optimtool_diagnoistics()

idmtools_calibra.plotters.separatrix_bhm_plotter module

class idmtools_calibra.plotters.separatrix_bhm_plotter.SeparatrixBHMPlotter

Bases: BasePlotter

make_prediction_grid()

cleanup()

visualize(iteration_state)

visualize_at_end_of_iteration()

visualize_at_start_of_iteration()

cleanup_plot(calib_manager)

idmtools_calibra.plotters.site_data_plotter module

class idmtools_calibra.plotters.site_data_plotter.SiteDataPlotter(combine_sites=True, num_to_plot=5, ll_all_name: str = 'LL_all.csv')

Bases: BasePlotter

property directory

get_site_analyzer(site_name, analyzer_name)

get_analyzer_data(iteration, site_name, analyzer_name)

visualize(iteration_state)

plot_analyzers(site_name, analyzer_names, samples)

plot_best(site_name, analyzer_name, samples)

plot_all(site_name, analyzer_name, samples, clim)

cleanup()

cleanup the existing plots :param calib_manager: :return:

cleanup_plot_by_analyzers(site, analyzers, samples)

cleanup the existing plots :param site: :param analyzers: :param samples: :return:

cleanup_plot_for_best(site_analyzer, samples)

cleanup the existing plots :param site_analyzer: :param samples: :return:

cleanup_plot_for_all(site_analyzer)

cleanup the existing plots :param site_analyzer: :return:

write_LL_csv(ll_all_name: str | None = None)

Write the LL_summary.csv with what is in the CalibManager

idmtools_calibra.resamplers package

Submodules

idmtools_calibra.resamplers.base_resampler module

class idmtools_calibra.resamplers.base_resampler.BaseResampler(calib_manager=None)

Bases: object

abstract resample(calibrated_points, selection_values, initial_calibration_points)

post_analysis(resampled_points, analyzer_results, from_resample=None)

set_calibration_manager(calib_manager: CalibManager)

resample_and_run(calibrated_points, resample_step, selection_values, initial_calibration_points)

Canonical entry method for using the resampler.

Parameters:

• calibrated_points – Calibrated Points

• resample_step –

• selection_values –

• initial_calibration_points –

Returns:

idmtools_calibra.resamplers.calibration_point module

class idmtools_calibra.resamplers.calibration_point.CalibrationPoint(parameters=None, likelihood=None)

Bases: object

DYNAMIC = 'dynamic'

STATIC = 'static'

ALL = 'all'

PARAMETER_TYPES = ['dynamic', 'static', 'all']

LIST = 'list'

SERIES = 'series'

NUMPY = 'numpy'

to_value_dict(parameter_type=None, include_likelihood=False)

Return the dict of dict containing {parameter_name:value} for this CalibrationPoint

get_attribute(key, parameter_type=None, as_type=None)

Returns the specified attribute of each CalibrationParameter as a list, ordered by parameter

Parameters:

• key –

• parameter_type –

• as_type –

Returns:

property parameter_names

get_parameter(name)

Relies on their being exactly one parameter with the given name.

Parameters:

name –

Returns:

to_dict()

Converts CalibrationPoint objects to a dictionary. Useful e.g. for dumping to a json file.

Returns:

a dict containing all needed information for recreating a CalibrationPoint object via from_dict()

classmethod from_dict(src_dict)

Inverse method of to_dict. Builds point from a Dictionary :param src_dict: a dictionary equivalent to one returned by to_dict()

Returns:

a CalibrationPoint object

to_dataframe(parameter_type=None)

write_point(filename)

class idmtools_calibra.resamplers.calibration_point.CalibrationParameter(name, min, max, value, mapTo, dynamic, guess)

Bases: object

classmethod from_dict(parameters)

classmethod from_calibration_parameter(parameter, value)

Create a new CalibrationParameter object from another one but with a new value. :param parameter: input CalibrationParameter to copy :param value: the value to override the copy with :return: a CalibrationParameter object

to_dict()

to_dataframe()

idmtools_calibra.resamplers.calibration_points module

class idmtools_calibra.resamplers.calibration_points.CalibrationPoints(points)

Bases: object

write(filename)

classmethod read(filename)

idmtools_calibra.resamplers.cramer_rao_resampler module

class idmtools_calibra.resamplers.cramer_rao_resampler.CramerRaoResampler(calib_manager=None, **kwargs)

Bases: BaseResampler

resample(calibrated_points, selection_values, initial_calibration_points)

Takes in a list of 1+ Point objects and returns method-specific resampled points as a list of Point objects The resultant Point objects should be copies of the input Points BUT with Value overridden on each, e.g.:

new_point = Point.copy(one_of_the_input_calibrated_points)
for param in new_point.list_params():
  new_point.set_param_value(param, value=SOME_NEW_VALUE)

Parameters:

calibrated_points – input points for this resampling method

Returns:

A list of resampled Point objects

post_analysis(resampled_points, analyzer_results, from_resample)

idmtools_calibra.resamplers.random_perturbation_resampler module

class idmtools_calibra.resamplers.random_perturbation_resampler.RandomPerturbationResampler(calib_manager=None, **kwargs)

Bases: BaseResampler

resample(calibrated_points, selection_values, initial_calibration_points)

Takes in a list of 1+ Point objects and returns method-specific resampled points as a list of Point objects The resultant Point objects should be copies of the input Points BUT with Value overridden on each, e.g.:

new_point = Point.copy(one_of_the_input_calibrated_points)
for param in new_point.list_params():
  new_point.set_param_value(param, value=SOME_NEW_VALUE)

Parameters:

• calibrated_points – input points for this resampling method

• selection_values –

• initial_calibration_points –

Returns:

A list of resampled Point objects

post_analysis(resampled_points, analyzer_results, from_resample=None)

Parameters:

• resampled_points –

• analyzer_results –

• from_resample –

Returns:

generate_perturbed_points(center_point) → DataFrame

given center and generate perturbed points

Parameters:

center_point – center point

Returns:

idmtools_calibra.utilities package

Subpackages

idmtools_calibra.utilities.parsers package

Submodules

idmtools_calibra.utilities.parsers.JSON module

idmtools_calibra.utilities.parsers.JSON.strip(pieces)

idmtools_calibra.utilities.parsers.JSON.parse(input_data, decoder=<json.decoder.JSONDecoder object>, pieces_mem=1024)

idmtools_calibra.utilities.parsers.JSON.json2dict(json_file, as_is=True, func=None)

idmtools_calibra.utilities.parsers.JSON.dict2json(filename, dict_content)

idmtools_calibra.utilities.parsers.bin2np module

idmtools_calibra.utilities.parsers.bin2np.filter_generator(src, func, sep=' ')

idmtools_calibra.utilities.parsers.bin2np.bin2np(src, func=None, sep=' ')

idmtools_calibra.utilities.parsers.dict2pd module

idmtools_calibra.utilities.parsers.dict2pd.dict2pd(d)

idmtools_calibra.utilities.parsers.malaria_summary module

idmtools_calibra.utilities.parsers.malaria_summary.summary_channel_to_pandas(data, channel)

A function to return a hierarchical binned pandas.Series for a specified MalariaSummaryReport.json channel :param data: parsed data from summary report :param channel: channel in summary report :return: pd.Series with MultiIndex binning taken from summary metadata

idmtools_calibra.utilities.parsers.malaria_summary.json_to_pandas(channel_data, bins, channel=None)

A function to convert nested array channel data from a json file to a pandas.Series with the specified MultiIndex binning.

idmtools_calibra.utilities.parsers.malaria_summary.get_grouping_for_summary_channel(data, channel)

A function to find the grouping to which a channel belongs in MalariaSummaryReport.json :param data: parsed data from summary report :param channel: channel to find :return: grouping or exception if not found

Example:

>>> get_grouping_for_summary_channel(data, channel='Average Population by Age Bin')
'DataByTimeAndAgeBins'

idmtools_calibra.utilities.parsers.malaria_summary.get_bins_for_summary_grouping(data, grouping)

A function to get the dimensions and binning of data for a specified MalariaSummaryReport.json grouping :param data: parsed data from summary report :param grouping: group name :return: an OrderedDict of dimensions and bins

Example:

>>> get_bins_for_summary_grouping(data, grouping='DataByTimeAndAgeBins')
OrderedDict([('Time', [31, 61, 92, ..., 1095]), ('Age Bin', [0, 10, 20, ..., 1000])])

idmtools_calibra.utilities.parsers.np2json module

class idmtools_calibra.utilities.parsers.np2json.NumpyAwareJSONEncoder(*, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, sort_keys=False, indent=None, separators=None, default=None)

Bases: JSONEncoder

default(obj)

Implement this method in a subclass such that it returns a serializable object for o, or calls the base implementation (to raise a TypeError).

For example, to support arbitrary iterators, you could implement default like this:

def default(self, o):
    try:
        iterable = iter(o)
    except TypeError:
        pass
    else:
        return list(iterable)
    # Let the base class default method raise the TypeError
    return JSONEncoder.default(self, o)

idmtools_calibra.utilities.parsers.np2json.np2json(arr, output='data.json')

idmtools_calibra.utilities.parsers.np2pd module

idmtools_calibra.utilities.parsers.np2pd.np2pd(arr, to_series=False)

idmtools_calibra.utilities.parsers.pd2csv module

idmtools_calibra.utilities.parsers.pd2csv.pd2csv(df, dst='dataframe.csv', chunksize=10)

Submodules

idmtools_calibra.utilities.display module

idmtools_calibra.utilities.display.on_off(test)

idmtools_calibra.utilities.display.pluralize(i, unit='s')

idmtools_calibra.utilities.display.verbose_timedelta(delta)

idmtools_calibra.utilities.distro module

The distro package (distro stands for Linux Distribution) provides information about the Linux distribution it runs on, such as a reliable machine-readable distro ID, or version information.

It is the recommended replacement for Python’s original platform.linux_distribution() function, but it provides much more functionality. An alternative implementation became necessary because Python 3.5 deprecated this function, and Python 3.8 will remove it altogether. Its predecessor function platform.dist() was already deprecated since Python 2.6 and will also be removed in Python 3.8. Still, there are many cases in which access to OS distribution information is needed. See Python issue 1322 for more information.

idmtools_calibra.utilities.distro.NORMALIZED_OS_ID = {}

Translation table for normalizing the “ID” attribute defined in os-release files, for use by the distro.id() method.

• Key: Value as defined in the os-release file, translated to lower case, with blanks translated to underscores.

• Value: Normalized value.

idmtools_calibra.utilities.distro.NORMALIZED_LSB_ID = {'enterpriseenterprise': 'oracle', 'redhatenterpriseserver': 'rhel', 'redhatenterpriseworkstation': 'rhel'}

Translation table for normalizing the “Distributor ID” attribute returned by the lsb_release command, for use by the distro.id() method.

• Key: Value as returned by the lsb_release command, translated to lower case, with blanks translated to underscores.

• Value: Normalized value.

idmtools_calibra.utilities.distro.NORMALIZED_DISTRO_ID = {'redhat': 'rhel'}

Translation table for normalizing the distro ID derived from the file name of distro release files, for use by the distro.id() method.

• Key: Value as derived from the file name of a distro release file, translated to lower case, with blanks translated to underscores.

• Value: Normalized value.

idmtools_calibra.utilities.distro.linux_distribution(full_distribution_name=True)

Return information about the current OS distribution as a tuple (id_name, version, codename) with items as follows:

• id_name: If full_distribution_name is false, the result of distro.id(). Otherwise, the result of distro.name().

• version: The result of distro.version().

• codename: The result of distro.codename().

The interface of this function is compatible with the original platform.linux_distribution() function, supporting a subset of its parameters.

The data it returns may not exactly be the same, because it uses more data sources than the original function, and that may lead to different data if the OS distribution is not consistent across multiple data sources it provides (there are indeed such distributions …).

Another reason for differences is the fact that the distro.id() method normalizes the distro ID string to a reliable machine-readable value for a number of popular OS distributions.

idmtools_calibra.utilities.distro.id()

Return the distro ID of the current distribution, as a machine-readable string.

For a number of OS distributions, the returned distro ID value is reliable, in the sense that it is documented and that it does not change across releases of the distribution.

This package maintains the following reliable distro ID values:

Distro ID	Distribution
“ubuntu”	Ubuntu
“debian”	Debian
“rhel”	RedHat Enterprise Linux
“centos”	CentOS
“fedora”	Fedora
“sles”	SUSE Linux Enterprise Server
“opensuse”	openSUSE
“amazon”	Amazon Linux
“arch”	Arch Linux
“cloudlinux”	CloudLinux OS
“exherbo”	Exherbo Linux
“gentoo”	GenToo Linux
“ibm_powerkvm”	IBM PowerKVM
“kvmibm”	KVM for IBM z Systems
“linuxmint”	Linux Mint
“mageia”	Mageia
“mandriva”	Mandriva Linux
“parallels”	Parallels
“pidora”	Pidora
“raspbian”	Raspbian
“oracle”	Oracle Linux (and Oracle Enterprise Linux)
“scientific”	Scientific Linux
“slackware”	Slackware
“xenserver”	XenServer
“openbsd”	OpenBSD
“netbsd”	NetBSD
“freebsd”	FreeBSD

If you have a need to get distros for reliable IDs added into this set, or if you find that the distro.id() function returns a different distro ID for one of the listed distros, please create an issue in the distro issue tracker.

Lookup hierarchy and transformations:

First, the ID is obtained from the following sources, in the specified order. The first available and non-empty value is used:

• the value of the “ID” attribute of the os-release file,

• the value of the “Distributor ID” attribute returned by the lsb_release command,

• the first part of the file name of the distro release file,

The so determined ID value then passes the following transformations, before it is returned by this method:

• it is translated to lower case,

• blanks (which should not be there anyway) are translated to underscores,

• a normalization of the ID is performed, based upon normalization tables. The purpose of this normalization is to ensure that the ID is as reliable as possible, even across incompatible changes in the OS distributions. A common reason for an incompatible change is the addition of an os-release file, or the addition of the lsb_release command, with ID values that differ from what was previously determined from the distro release file name.

idmtools_calibra.utilities.distro.name(pretty=False)

Return the name of the current OS distribution, as a human-readable string.

If pretty is false, the name is returned without version or codename. (e.g. “CentOS Linux”)

If pretty is true, the version and codename are appended. (e.g. “CentOS Linux 7.1.1503 (Core)”)

Lookup hierarchy:

The name is obtained from the following sources, in the specified order. The first available and non-empty value is used:

• If pretty is false:

  • the value of the “NAME” attribute of the os-release file,

  • the value of the “Distributor ID” attribute returned by the lsb_release command,

  • the value of the “<name>” field of the distro release file.

• If pretty is true:

  • the value of the “PRETTY_NAME” attribute of the os-release file,

  • the value of the “Description” attribute returned by the lsb_release command,

  • the value of the “<name>” field of the distro release file, appended with the value of the pretty version (“<version_id>” and “<codename>” fields) of the distro release file, if available.

idmtools_calibra.utilities.distro.version(pretty=False, best=False)

Return the version of the current OS distribution, as a human-readable string.

If pretty is false, the version is returned without codename (e.g. “7.0”).

If pretty is true, the codename in parenthesis is appended, if the codename is non-empty (e.g. “7.0 (Maipo)”).

Some distributions provide version numbers with different precisions in the different sources of distribution information. Examining the different sources in a fixed priority order does not always yield the most precise version (e.g. for Debian 8.2, or CentOS 7.1).

The best parameter can be used to control the approach for the returned version:

If best is false, the first non-empty version number in priority order of the examined sources is returned.

If best is true, the most precise version number out of all examined sources is returned.

Lookup hierarchy:

In all cases, the version number is obtained from the following sources. If best is false, this order represents the priority order:

• the value of the “VERSION_ID” attribute of the os-release file,

• the value of the “Release” attribute returned by the lsb_release command,

• the version number parsed from the “<version_id>” field of the first line of the distro release file,

• the version number parsed from the “PRETTY_NAME” attribute of the os-release file, if it follows the format of the distro release files.

• the version number parsed from the “Description” attribute returned by the lsb_release command, if it follows the format of the distro release files.

idmtools_calibra.utilities.distro.version_parts(best=False)

Return the version of the current OS distribution as a tuple (major, minor, build_number) with items as follows:

• major: The result of distro.major_version().

• minor: The result of distro.minor_version().

• build_number: The result of distro.build_number().

For a description of the best parameter, see the distro.version() method.

idmtools_calibra.utilities.distro.major_version(best=False)

Return the major version of the current OS distribution, as a string, if provided. Otherwise, the empty string is returned. The major version is the first part of the dot-separated version string.

For a description of the best parameter, see the distro.version() method.

idmtools_calibra.utilities.distro.minor_version(best=False)

Return the minor version of the current OS distribution, as a string, if provided. Otherwise, the empty string is returned. The minor version is the second part of the dot-separated version string.

For a description of the best parameter, see the distro.version() method.

idmtools_calibra.utilities.distro.build_number(best=False)

Return the build number of the current OS distribution, as a string, if provided. Otherwise, the empty string is returned. The build number is the third part of the dot-separated version string.

For a description of the best parameter, see the distro.version() method.

idmtools_calibra.utilities.distro.like()

Return a space-separated list of distro IDs of distributions that are closely related to the current OS distribution in regards to packaging and programming interfaces, for example distributions the current distribution is a derivative from.

Lookup hierarchy:

This information item is only provided by the os-release file. For details, see the description of the “ID_LIKE” attribute in the os-release man page.

idmtools_calibra.utilities.distro.codename()

Return the codename for the release of the current OS distribution, as a string.

If the distribution does not have a codename, an empty string is returned.

Note that the returned codename is not always really a codename. For example, openSUSE returns “x86_64”. This function does not handle such cases in any special way and just returns the string it finds, if any.

Lookup hierarchy:

• the codename within the “VERSION” attribute of the os-release file, if provided,

• the value of the “Codename” attribute returned by the lsb_release command,

• the value of the “<codename>” field of the distro release file.

idmtools_calibra.utilities.distro.info(pretty=False, best=False)

Return certain machine-readable information items about the current OS distribution in a dictionary, as shown in the following example:

{
    'id': 'rhel',
    'version': '7.0',
    'version_parts': {
        'major': '7',
        'minor': '0',
        'build_number': ''
    },
    'like': 'fedora',
    'codename': 'Maipo'
}

The dictionary structure and keys are always the same, regardless of which information items are available in the underlying data sources. The values for the various keys are as follows:

• id: The result of distro.id().

• version: The result of distro.version().

• version_parts -> major: The result of distro.major_version().

• version_parts -> minor: The result of distro.minor_version().

• version_parts -> build_number: The result of distro.build_number().

• like: The result of distro.like().

• codename: The result of distro.codename().

For a description of the pretty and best parameters, see the distro.version() method.

idmtools_calibra.utilities.distro.os_release_info()

Return a dictionary containing key-value pairs for the information items from the os-release file data source of the current OS distribution.

See os-release file for details about these information items.

idmtools_calibra.utilities.distro.lsb_release_info()

Return a dictionary containing key-value pairs for the information items from the lsb_release command data source of the current OS distribution.

See lsb-release command output for details about these information items.

idmtools_calibra.utilities.distro.distro_release_info()

Return a dictionary containing key-value pairs for the information items from the distro release file data source of the current OS distribution.

See distro release file for details about these information items.

idmtools_calibra.utilities.distro.uname_info()

Return a dictionary containing key-value pairs for the information items from the distro release file data source of the current OS distribution.

idmtools_calibra.utilities.distro.os_release_attr(attribute)

Return a single named information item from the os-release file data source of the current OS distribution.

Parameters:

• attribute (string): Key of the information item.

Returns:

• (string): Value of the information item, if the item exists. The empty string, if the item does not exist.

See os-release file for details about these information items.

idmtools_calibra.utilities.distro.lsb_release_attr(attribute)

Return a single named information item from the lsb_release command output data source of the current OS distribution.

Parameters:

• attribute (string): Key of the information item.

Returns:

• (string): Value of the information item, if the item exists. The empty string, if the item does not exist.

See lsb-release command output for details about these information items.

idmtools_calibra.utilities.distro.distro_release_attr(attribute)

Return a single named information item from the distro release file data source of the current OS distribution.

Parameters:

• attribute (string): Key of the information item.

Returns:

• (string): Value of the information item, if the item exists. The empty string, if the item does not exist.

See distro release file for details about these information items.

idmtools_calibra.utilities.distro.uname_attr(attribute)

Return a single named information item from the distro release file data source of the current OS distribution.

Parameters:

• attribute (string): Key of the information item.

Returns:

• (string): Value of the information item, if the item exists.

  The empty string, if the item does not exist.

class idmtools_calibra.utilities.distro.cached_property(f)

Bases: object

A version of @property which caches the value. On access, it calls the underlying function and sets the value in __dict__ so future accesses will not re-call the property.

class idmtools_calibra.utilities.distro.LinuxDistribution(include_lsb=True, os_release_file='', distro_release_file='', include_uname=True)

Bases: object

Provides information about a OS distribution.

This package creates a private module-global instance of this class with default initialization arguments, that is used by the consolidated accessor functions and single source accessor functions. By using default initialization arguments, that module-global instance returns data about the current OS distribution (i.e. the distro this package runs on).

Normally, it is not necessary to create additional instances of this class. However, in situations where control is needed over the exact data sources that are used, instances of this class can be created with a specific distro release file, or a specific os-release file, or without invoking the lsb_release command.

linux_distribution(full_distribution_name=True)

Return information about the OS distribution that is compatible with Python’s platform.linux_distribution(), supporting a subset of its parameters.

For details, see distro.linux_distribution().

id()

Return the distro ID of the OS distribution, as a string.

For details, see distro.id().

name(pretty=False)

Return the name of the OS distribution, as a string.

For details, see distro.name().

version(pretty=False, best=False)

Return the version of the OS distribution, as a string.

For details, see distro.version().

version_parts(best=False)

Return the version of the OS distribution, as a tuple of version numbers.

For details, see distro.version_parts().

major_version(best=False)

Return the major version number of the current distribution.

For details, see distro.major_version().

minor_version(best=False)

Return the minor version number of the current distribution.

For details, see distro.minor_version().

build_number(best=False)

Return the build number of the current distribution.

For details, see distro.build_number().

like()

Return the IDs of distributions that are like the OS distribution.

For details, see distro.like().

codename()

Return the codename of the OS distribution.

For details, see distro.codename().

info(pretty=False, best=False)

Return certain machine-readable information about the OS distribution.

For details, see distro.info().

os_release_info()

Return a dictionary containing key-value pairs for the information items from the os-release file data source of the OS distribution.

For details, see distro.os_release_info().

lsb_release_info()

Return a dictionary containing key-value pairs for the information items from the lsb_release command data source of the OS distribution.

For details, see distro.lsb_release_info().

distro_release_info()

Return a dictionary containing key-value pairs for the information items from the distro release file data source of the OS distribution.

For details, see distro.distro_release_info().

uname_info()

Return a dictionary containing key-value pairs for the information items from the uname command data source of the OS distribution.

For details, see distro.uname_info().

os_release_attr(attribute)

Return a single named information item from the os-release file data source of the OS distribution.

For details, see distro.os_release_attr().

lsb_release_attr(attribute)

Return a single named information item from the lsb_release command output data source of the OS distribution.

For details, see distro.lsb_release_attr().

distro_release_attr(attribute)

Return a single named information item from the distro release file data source of the OS distribution.

For details, see distro.distro_release_attr().

uname_attr(attribute)

Return a single named information item from the uname command output data source of the OS distribution.

For details, see distro.uname_release_attr().

idmtools_calibra.utilities.distro.main()

idmtools_calibra.utilities.distro.test()

idmtools_calibra.utilities.encoding module

class idmtools_calibra.utilities.encoding.NumpyEncoder(*, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, sort_keys=False, indent=None, separators=None, default=None)

Bases: JSONEncoder

default(obj)

If input object is an ndarray it will be converted into a dict holding dtype, shape and the data, base64 encoded.

class idmtools_calibra.utilities.encoding.GeneralEncoder(*, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, sort_keys=False, indent=None, separators=None, default=None)

Bases: NumpyEncoder

default(obj)

If input object is an ndarray it will be converted into a dict holding dtype, shape and the data, base64 encoded.

idmtools_calibra.utilities.encoding.json_numpy_obj_hook(dct)

Decodes a previously encoded numpy ndarray with proper shape and dtype.

Parameters:

dct – (dict) json encoded ndarray

Returns:

(ndarray) if input was an encoded ndarray

idmtools_calibra.utilities.encoding.cast_number(val)

Try casting the value to float/int returns str if cannot :param val: the value to cast :return: value casted

idmtools_calibra.utilities.helper module

idmtools_calibra.utilities.helper.validate_exp_name(exp_name)

idmtools_calibra.utilities.helper.json_to_pandas(channel_data, bins, channel=None)

A function to convert nested array channel data from a json file to a pandas.Series with the specified MultiIndex binning.

idmtools_calibra.utilities.ll_all_generator module

idmtools_calibra.utilities.ll_all_generator.generate_ll_all(calib_manager, num_to_plot=5, iteration=None, ll_all_name=None)

idmtools_calibra.utilities.ll_calculators module

idmtools_calibra.utilities.ll_calculators.dirichlet_multinomial_pandas(df)

idmtools_calibra.utilities.ll_calculators.gamma_poisson_pandas(df)

idmtools_calibra.utilities.ll_calculators.beta_binomial_pandas(df)

idmtools_calibra.utilities.ll_calculators.dirichlet_multinomial(raw_data, sim_data)

idmtools_calibra.utilities.ll_calculators.dirichlet_single(raw_data, sim_data)

idmtools_calibra.utilities.ll_calculators.beta_binomial(raw_nobs, sim_nobs, raw_data, sim_data, return_mean=True)

idmtools_calibra.utilities.ll_calculators.gamma_poisson(raw_nobs, sim_nobs, raw_data, sim_data, return_mean=True)

idmtools_calibra.utilities.ll_calculators.euclidean_distance(raw_data, sim_data)

idmtools_calibra.utilities.ll_calculators.euclidean_distance_pandas(df)

idmtools_calibra.utilities.ll_calculators.weighted_squares(raw_data, sim_data)

idmtools_calibra.utilities.mod_fn module

class idmtools_calibra.utilities.mod_fn.ModFn(func, *args, **kwargs)

Bases: object

idmtools_calibra.utilities.parameter_set module

exception idmtools_calibra.utilities.parameter_set.NaNDetectedError

Bases: ValueError

class idmtools_calibra.utilities.parameter_set.ParameterSet(param_dict, iteration_number=None, run_number=None, sim_id=None, likelihood=None, sublikelihoods=None)

Bases: object

to_dict()

classmethod from_dict(source_dict)

idmtools_calibra.utilities.prior module

class idmtools_calibra.utilities.prior.SampleRange(range_type, range_min, range_max)

Bases: object

Container for min, max of a range and a type of sampling to use

Parameters:

• range_type – Type of sampling within range. Supported values: linear, log, linear_int

• range_min – Minimum of sampling range.

• range_max – Maximum of sampling range.

get_sample_function()

Converts sample-range variables into scipy.stats frozen function :return: Frozen scipy.stats function

get_bins(n)

get_xlim()

is_log()

is_int()

class idmtools_calibra.utilities.prior.SampleFunctionContainer(function, sample_range=None)

Bases: object

Container for a frozen function and optionally its associated sample-range properties

classmethod from_range(sample_range)

classmethod from_tuple(*args)

get_even_spaced_samples(n)

Returns an evenly spaced sampling of the percent-point function (inverse CDF) :param n: number of evenly spaced samples

pdf(X)

Wrapper for contained function pdf with check for discrete distributions :return:

class idmtools_calibra.utilities.prior.MultiVariatePrior(functions, params=[], ranges=[], name=None)

Bases: object

Multi-variate wrapper exposing same interfaces as scipy.stats functions, i.e. pdf and rvs

Different dimensions are drawn independently from the univariate distributions.

Parameters:

• sample_functions – list of scipy.stats frozen functions

• params – list of parameter names associated with functions (optional)

• ranges – list of SampleRange objects associated with functions (optional)

• name – name of MultiVariatePrior object (optional)

property functions

property params

property ndim

classmethod by_range(**param_sample_ranges)

Builds multi-variate wrapper from keyword arguments of parameter names to SampleRange (min, max, type)

Parameters:

param_sample_ranges – keyword arguments of parameter names to SampleRange tuple

Returns:

MultiVariatePrior instance

An example usage:

> prior = MultiVariatePrior.by_range(
      MSP1_Merozoite_Kill_Fraction=('linear', 0.4, 0.7),
      Max_Individual_Infections=('linear_int', 3, 8),
      Base_Gametocyte_Production_Rate=('log', 0.001, 0.5))

classmethod by_param(**param_sample_functions)

Builds multi-variate wrapper from keyword arguments of parameter names to univariate frozen functions

Parameters:

param_sample_functions – keyword arguments of parameter names to univariate object supporting pdf and rvs interfaces

Returns:

MultiVariatePrior instance

An example usage:

> from scipy.stats import uniform
> prior = MultiVariatePrior.by_param(
      MSP1_Merozoite_Kill_Fraction=uniform(loc=0.4, scale=0.3),  # from 0.4 to 0.7
      Nonspecific_Antigenicity_Factor=uniform(loc=0.1, scale=0.8))  # from 0.1 to 0.9

pdf(X)

Returns product of individual component function PDFs at each input point.

Parameters:

X – array of points, where each point is an array of correct dimension.

rvs(size=1)

Returns an array of random points, where each point is sampled randomly in its component dimensions.

Parameters:

size – the number of random points to sample.

lhs(size=1)

Returns a Latin Hypercube sample, drawn from the sampling ranges of the component dimensions :param size: the number of random points to sample

to_dataframe(param_value_array)

Transforms an array of parameter-value arrays to a pandas.DataFrame with appropriate column names

to_dict(param_point)

Transforms an individual point in parameter space to a dictionary of parameter names to values. Also will round parameters where the range_type requires integer-only values.

idmtools_calibra.utilities.resume_manager module

idmtools_calibra.utilities.resume_manager.status_to_iter_step(status: StatusPoint)

class idmtools_calibra.utilities.resume_manager.ResumeManager(calib_manager: CalibManager, iteration: int | None = None, iter_step: str | None = None, max_iterations: int | None = None, loop: bool = True, backup: bool = False, dry_run: bool = False)

Bases: object

Manages the creation, execution, and resumption of multi-iteration a calibration suite. Each iteration spawns a new ExperimentManager to configure and commission either local or HPC simulations for a set of random seeds, sample points, and site configurations.

initialize()

prepare calib_manager and iteration state for resume

• restore calib_manager

• restore iteration state

• validate iteration

• validate iter_step

resume()

Call calib_manager.run_iterations to start resume action

check_location()

• Handle the case: resume on different environments

• Handle environment change case: may resume from commission instead

adjust_iteration()

Validate iteration against latest_iteration return adjusted iteration

adjust_iteration_step()

Validate iter_step

restore_calib_manager()

Restore calib_manager

restore_iteration_state()

Restore IterationState

restore_ll_all()

backup_calibration()

Backup CalibManager.json for resume action

Submodules

idmtools_calibra.calib_base_app module

idmtools_calibra.calib_base_app.constrain_sample(sample)

This function is called on every samples and allow the user to edit them before they are passed to the map_sample_to_model_input function. It is useful to round some parameters as demonstrated below. #Can do much more here, e.g. for Clinical Fever Threshold High < MSP1 Merozoite Kill Fraction if ‘Clinical Fever Threshold High’ and “MSP1 Merozoite Kill Fraction” in sample: sample[‘Clinical Fever Threshold High’] = min( sample[‘Clinical Fever Threshold High’], sample[‘MSP1 Merozoite Kill Fraction’] ). You can omit this function by not specifying it in the OptimTool constructor call below. :param sample: The sample coming from the next point algorithm

Returns: The sample with constrained values

idmtools_calibra.calib_base_app.map_sample_to_model_input(simulation, sample)

This method maps the samples generated by the next point algorithm to the model inputs (via the simulation object). All parameters specified for dynamic calibration (above) that do not have a MapTo value must have associated mapping logic in this method. :param simulation: idmtools simulation :param sample: The sample containing a values for all the params. e.g. {‘Clinical Fever Threshold High’:1, … }

Returns: A dictionary containing the tags that will be attached to the simulation

idmtools_calibra.calib_base_app.init(settings, site, task=None, platform=None)

idmtools_calibra.calib_base_app.go(calib_manager, **kwargs)

idmtools_calibra.calib_manager module

idmtools_calibra.calib_manager.set_parameter_sweep_callback(simulation: Simulation, param: str, value: Any) → Dict[str, Any]

Convenience callback for sweeps

Parameters:

• simulation – Simulation we are updating

• param – Parameter

• value – Value

Returns:

Tags to set on simulation

class idmtools_calibra.calib_manager.SampleIndexWrapper(map_sample_to_model_input_fn)

Bases: object

Wrapper for a SimConfigBuilder-modifying function to add metadata on sample-point index when called in a iteration over sample points

class idmtools_calibra.calib_manager.CalibManager(task: ITask, map_sample_to_model_input_fn, sites: List[CalibSite], next_point: NextPointAlgorithm, platform: IPlatform | None = None, name: str = 'calib_test', sim_runs_per_param_set: int = 1, max_iterations: int = 5, plotters: List[BasePlotter] | None = None, map_replicates_callback=None, directory='.')

Bases: object

Manages the creation, execution, and resumption of multi-iteration a calibration suite. Each iteration spawns a new ExperimentManager to configure and commission either local or HPC simulations for a set of random seeds, sample points, and site configurations.

classmethod open_for_reading(calibration_directory)

property suite_id

property iteration

run_calibration(**kwargs)

Create and run a complete multi-iteration calibration suite. kwargs supports the following optional parameters: :param resume: bool, default=False, flag required for calibration resume :param iteration: int, default=None, from which iteration to resume :param iter_step: str, default=None, from which calibration step to resume. support values: ‘commission’, ‘analyze’, plot’ and ‘next_point’ :param loop: bool, default=True, if like to continue to next iteration :param max_iterations, int, default=None, user can override the max_iterations defined in calib_manager: :param backup: bool, default=False, if like to backup Calibration.json :param dry_run: bool, default=False, if like to really execute resume action :param directory: str, default=None, calibration directory

Returns: None

run_iterations(iteration: int = 0, max_iterations: int | None = None, loop: bool = True)

Run iterations in a loop :param iteration: the # of iterations :param max_iterations: max iterations :param loop: if or not continue iteration loop

Returns: None

post_iteration()

set_experiment_builder_function(exp_builder_function)

Set the experiment builder to a predefined one, such that exp_builder_func will return it :param exp_builder_function: an experiment builder object

Returns: None

set_map_replicates_callback(map_replicates_callback)

This sets the maps replicate callback. This callback should take a value parameter that is the value of the replicate.

Normally you would want to map this to a value in the config :param map_replicates_callback:

Returns:

default_experiment_builder_function(next_params, n_replicates: int | None = None) → SimulationBuilder

Defines the function that builds the experiment for each iteration of a calibration run

Parameters:

• next_params – The next parameters to run

• n_replicates – Number of replicates

Returns:

Simulation Builder

create_iteration_state(iteration)

Create iteration state :param iteration: the # of the iteration

Returns: created IterationState

create_calibration()

Create the working directory for a new calibration. Cache the relevant suite-level information to allow re-initializing this instance.

finalize_calibration()

Get the final samples from the next point algorithm.

cache_calibration(**kwargs)

Cache information about the CalibManager that is needed to resume after an interruption. N.B. This is not currently the complete state, some of which relies on nested and frozen functions. As such, the ‘resume’ logic relies on the existence of the original configuration script. :param **kwargs: extra info

Returns: None

backup_calibration()

Backup CalibManager.json for resume action

serialize_results()

resume_calibration(**kwargs)

delete()

cleanup()

Cleanup the current calibration - Delete the result directory - If LOCAL -> also delete the simulations

read_calib_data(force=False)

property calibration_path

property analyzer_list

property required_components

iteration_directory()

state_for_iteration(iteration)

param_names()

site_analyzer_names()

get_last_iteration()

Determines the last (most recent) completed iteration number. Returns: the last completed iteration number as an int

get_parameter_sets_with_likelihoods()

Returns: a list of ParameterSet objects.

idmtools_calibra.calib_site module

class idmtools_calibra.calib_site.SiteFunctions(name, setup_functions, verbose=False)

Bases: object

A helper to take a set of bare SimConfigBuilder-modifying functions and combine them into a single function of the same format

set_calibration_site(task)

N.B. The name of this function is chosen to ensure it is applied first by ModBuilder.set_mods and other aspects can be over-ridden as needed by sample-point modifications.

class idmtools_calibra.calib_site.CalibSite(name)

Bases: object

A class to represent the base behavior of a calibration site

abstract get_reference_data(reference_type)

Callback function for derived classes to pass site-specific reference data that is requested by analyzers by the relevant reference_type.

abstract get_analyzers()

Derived classes return a list of BaseComparisonAnalyzer instances that have been passed a reference to the CalibSite for site-specific analyzer setup.

abstract get_setup_functions()

Derived classes return a list of functions to apply site-specific modifications to the base configuration. These are combined into a single function using the SiteFunctions helper class in the CalibSite constructor.

idmtools_calibra.iteration_state module

class idmtools_calibra.iteration_state.IterationState(**kwargs)

Bases: object

Holds the settings, parameters, simulation state, analysis results, etc. for one calibtool iteration.

Allows for the resumption or extension of existing CalibManager instances from an arbitrary point in the iterative process.

property status

update(**kwargs)

restore_results(iteration)

Restore summary results from serialized state. :param iteration: the # of iteration

Returns:

run()

starting_step()

commission_step()

analyze_step()

plotting_step()

next_point_step()

commission_iteration(next_params)

Commission an experiment of simulations constructed from a list of combinations of random seeds, calibration sites, and the next sample points. Cache the relevant experiment and simulation information to the IterationState. :param next_params: the next sample

Returns: None

plot_iteration()

analyze_iteration()

Analyze the output of completed simulations by using the relevant analyzers by site. Cache the results that are returned by those analyzers.

wait_for_finished(init_sleep=1.0, sleep_time=30)

cancel()

property iteration_directory

property iteration_file

property param_names

finished()

The next-point algorithm has reached its termination condition.

classmethod from_file(filepath)

to_file()

classmethod restore_state(iteration)

Restore IterationState

set_samples_for_iteration(samples, next_point)

save()

Cache information about the IterationState that is needed to resume after an interruption. If resuming from an existing iteration, also copy to backup the initial cached state.

get_parameter_sets_with_likelihoods()

idmtools_calibra.process_state module

class idmtools_calibra.process_state.StatusPoint(value)

Bases: Enum

An enumeration.

iteration_start = 0

commission = 1

running = 2

analyze = 3

plot = 4

next_point = 5

done = 6

idmtools_calibra.resample_manager module

class idmtools_calibra.resample_manager.ResampleManager(steps, calibration_manager: CalibManager, restart_at_step=None)

Bases: object

resample_and_run()

write_restart(step, selection_values)

load_restart(step)

get_calibrated_points()

Retrieve information about the most recent (final completed) iteration’s calibrated point, merging from the final IterationState.json and CalibManager.json.

Returns:

idmtools_calibra.rmse_site module

class idmtools_calibra.rmse_site.RMSESiteSingleChannel(name, reference_sources={'just_one': 'reference/output.csv'})

Bases: CalibSite

get_reference_data(reference_type=None)

Callback function for derived classes to pass site-specific reference data that is requested by analyzers by the relevant reference_type.

get_analyzers()

Derived classes return a list of BaseComparisonAnalyzer instances that have been passed a reference to the CalibSite for site-specific analyzer setup.

get_setup_functions()

Derived classes return a list of functions to apply site-specific modifications to the base configuration. These are combined into a single function using the SiteFunctions helper class in the CalibSite constructor.

idmtools_calibra.singularity_json_python_task module

class idmtools_calibra.singularity_json_python_task.SingularityJSONConfiguredPythonTask(command: Union[str, idmtools.entities.command_line.CommandLine] = <property object at 0x7f21207b0950>, platform_requirements: Set[idmtools.entities.platform_requirements.PlatformRequirements] = <factory>, _ITask__pre_creation_hooks: List[Callable[[Union[ForwardRef('Simulation'), ForwardRef('IWorkflowItem')], ForwardRef('IPlatform')], NoReturn]] = <factory>, _ITask__post_creation_hooks: List[Callable[[Union[ForwardRef('Simulation'), ForwardRef('IWorkflowItem')], ForwardRef('IPlatform')], NoReturn]] = <factory>, common_assets: idmtools.assets.asset_collection.AssetCollection = <factory>, transient_assets: idmtools.assets.asset_collection.AssetCollection = <factory>, script_path: str = None, python_path: str = 'python', parameters: dict = <factory>, envelope: str = None, config_file_name: str = 'config.json', is_config_common: bool = False, configfile_argument: Optional[str] = '--config', command_line_argument_no_filename: bool = False, provided_command: Optional[idmtools.entities.command_line.CommandLine] = <factory>)

Bases: JSONConfiguredPythonTask

provided_command: CommandLine | None

pre_creation(parent, platform)

Pre-creation.

Parameters:

• parent – Parent of task

• platform – Platform Python Script is being executed on

Returns:

None

See Also

idmtools_models.json_configured_task.JSONConfiguredTask.pre_creation() idmtools_models.python.python_task.PythonTask.pre_creation()

static set_parameter_sweep_callback(simulation: Simulation, param: str, value: Any) → Dict[str, Any]

Convenience callback for sweeps

Parameters:

• simulation – Simulation we are updating

• param – Parameter

• value – Value

Returns:

Tags to set on simulation

classmethod set_param_partial(parameter: str)