Calculate comprehensive convergence diagnostics with status indicators

Centralized function for computing all convergence diagnostics and status indicators for BFRS calibration results. Takes metrics and targets as input, calculates pass/warn/fail status for each metric using consistent thresholds, and aggregates to an overall convergence status.

Usage

calc_convergence_diagnostics(
  n_total,
  n_successful,
  n_retained,
  n_best_subset,
  ess_best,
  A_best,
  cvw_best,
  percentile_used,
  convergence_tier,
  param_ess_results = NULL,
  target_ess_best = 300,
  target_A_best = 0.95,
  target_cvw_best = 0.5,
  target_max_best_subset = 1000,
  target_ess_param = 300,
  target_ess_param_prop = 0.95,
  ess_method = c("kish", "perplexity"),
  temperature = 1,
  is_diagnostics = NULL,
  verbose = TRUE
)

Arguments

n_total: Integer total number of simulations attempted
n_successful: Integer number of simulations that completed successfully (finite likelihood)
n_retained: Integer number of simulations retained after outlier removal
n_best_subset: Integer number of simulations in the optimized best subset
ess_best: Numeric effective sample size within the best subset
A_best: Numeric agreement index (entropy-based) for the best subset
cvw_best: Numeric coefficient of variation of weights in the best subset
percentile_used: Numeric size of the best subset as a percentage of all n_total draws (e.g., 5.0 for the top 5%)
convergence_tier: Character string indicating which tier criteria were used (e.g., "tier_3", "fallback")
param_ess_results: Optional data frame with columns parameter and ess_marginal containing parameter-specific ESS values. If NULL, parameter ESS metrics are omitted.
target_ess_best: Numeric target for both subset size and ESS (default 300). Both B_size and ESS_B must be >= target_ess_best.
target_A_best: Numeric target agreement index (default 0.95)
target_cvw_best: Numeric target coefficient of variation (default 0.5)
target_max_best_subset: Numeric maximum best subset size in absolute count (default 1000). Replaced target_percentile_max for absolute count-based control.
target_ess_param: Numeric target ESS for individual parameters (default 300)
target_ess_param_prop: Numeric target proportion of parameters that must meet ESS threshold (default 0.95, meaning 95%)
ess_method: Character string specifying ESS calculation method: "kish" (default) or "perplexity". This should match the method used for all ESS calculations in the workflow.
temperature: Numeric temperature parameter used for weight scaling (default 1, for documentation purposes)
is_diagnostics: Optional list of exact importance-sampling diagnostics from calc_is_diagnostics, with elements best and all. Reported under $importance_sampling but never gated: these are untruncated-weight quantities, unlike the gated ESS_B.
verbose: Logical indicating whether to print diagnostic messages (default TRUE)

Value

A list with the following structure:

settings: List with optimization_tier, percentile_used, temperature, description
targets: List of target values with descriptions for each metric
metrics: List of metric values with descriptions and status indicators
summary: List with counts, tier info, and overall convergence status

The returned list is ready for JSON serialization and contains all information needed for visualization by plot_model_convergence_status.

Status Thresholds

Status indicators are calculated using multiplicative thresholds relative to targets:

For "higher is better" metrics (ESS, Agreement):

PASS: value >= target * pass_threshold
WARN: value >= target * warn_threshold but < pass_threshold
FAIL: value < target * warn_threshold

For "lower is better" metrics (CVw):

PASS: value <= target * pass_threshold
WARN: value <= target * warn_threshold but > pass_threshold
FAIL: value > target * warn_threshold

Default thresholds by metric:

ESS_B (target target_ess_best): pass=1.0 (100%), warn=0.8 (80%)
A_B: pass=0.9 (90%), warn=0.7 (70%)
CVw_B: pass=1.2 (120%), warn=2.0 (200%)
B_size (lower bound, target target_ess_best): pass=1.0 (100%), warn=0.5 (50%)
B_size_upper (upper cap, target target_max_best_subset, lower is better): pass when n_best_subset <= target, warn when it exceeds the cap by at most 20%, fail beyond that. run_MOSAIC() enforces the cap when it selects the subset, so in a normal run this is an invariant check.
Param ESS: pass=1.0 (100%), warn=0.8 (80%)

The subset percentile (percentile_used against target_max_best_subset / n_total * 100) carries the same information as B_size_upper on the same denominator, so its status is reported in summary$percentile_status but is not gated.

The function defaults for the targets (300 / 0.95 / 0.5) are not the run_MOSAIC() control defaults (ESS_best = 100, A_best = 0.70, CVw_best = 1.0); run_MOSAIC() always passes its targets explicitly.

Overall Status

The overall convergence status is determined by aggregating individual metric statuses:

FAIL: Any metric has status "fail"
WARN: No failures, but at least one metric has status "warn"
PASS: All metrics have status "pass"

Examples

if (FALSE) { # \dontrun{
# After running BFRS calibration and calculating metrics
diagnostics <- calc_convergence_diagnostics(
    n_total = 10000,
    n_successful = 9500,
    n_retained = 8500,
    n_best_subset = 500,
    ess_best = 280,
    A_best = 0.92,
    cvw_best = 0.55,
    percentile_used = 4.8,
    convergence_tier = "tier_3",
    param_ess_results = param_ess_df,
    target_ess_best = 300,
    target_A_best = 0.95,
    target_cvw_best = 0.5,
    target_max_best_subset = 1000,
    ess_method = "kish"
)

# Save to JSON. digits = NA preserves full numerical precision;
# the default digits = 4 silently rounds small diagnostic values
# (p-values, ESS ratios, etc.) to zero.
jsonlite::write_json(diagnostics, "convergence_diagnostics.json",
                     pretty = TRUE, auto_unbox = TRUE, digits = NA)

# Check overall status
print(diagnostics$summary$convergence_status)
} # }