Reporting and likelihood¶
This page explains how laser.cholera turns the simulation's internal compartmental state into the
observable time series that calibration and scoring actually compare against, and how those
observables are then folded into a single scalar log-likelihood. It assumes you have read
Model overview and are comfortable with the rough shape of an SEIRV step. By
the end you should be able to answer three questions: which random draws separate "true" cases
from "reported" cases, what the four-term likelihood ensemble actually measures, and where each
parameter on the reporting / scoring surface enters the math.
Reporting is a thinning of the truth¶
Every tick, the Infectious component samples a
fresh batch of new symptomatic incidences from the E -> Isym progression draw:
These are stored on model.patches.new_symptomatic[t+1] — they are the true incident
symptomatic infections in patch \(j\) on tick \(t+1\). The reporting transform then converts them into
the observable count reported_cases via two stochastic steps:
There are three knobs here, each doing a distinct job:
- The detection probability \(\rho\). A scalar in \([0, 1]\). Each truly-symptomatic case has an independent \(\rho\) chance of being detected. With \(\rho = 1\) every symptomatic case is detected (modulo the \(\chi\) inflation, below); with \(\rho = 0.1\) only ~10% are. This is a classical observation-process thinning.
- The reporting lag \(\delta_R^{\mathrm{cases}}\). An integer number of ticks. The binomial is drawn from the incidence at tick \(t - \delta_R^{\mathrm{cases}}\), not at tick \(t\). This is how the model represents the delay between case onset and the case appearing in surveillance totals.
- The healthcare-access modifier \(\chi^{\mathrm{eff}}_{j,t}\). A per-patch scalar in \((0, 1]\) that divides the detected count. Because it sits in the denominator, \(\chi^{\mathrm{eff}} < 1\) inflates reported cases above the binomial draw, and \(\chi^{\mathrm{eff}} = 1\) is the identity. The intuition: in an established outbreak, surveillance attention rises and the same underlying case gets logged in more streams; \(\chi\) is a coarse stand-in for that effect.
Reported deaths follow the identical pattern, with two substitutions: disease_deaths in place
of new_symptomatic, the scalar rho_deaths in place of rho, and the integer
delta_reporting_deaths in place of delta_reporting_cases. Critically, the death stream does
not divide by \(\chi\) — the implementation at
infectious.py:209-212 applies only the binomial
detection step. If you want a \(\chi\)-style inflation on deaths you would have to bake it into
rho_deaths.
The regime-switching factor \(\chi^{\mathrm{eff}}\)¶
The effective \(\chi\) is picked per patch, per tick, from a two-way switch:
The lagged symptomatic prevalence is compared against epidemic_threshold; this is the same
quantity used to decide whether the per-patch disease-mortality multiplier kicks in. To make the
reporting transform a no-op, set chi_endemic == chi_epidemic == 1.0. (To disable
regime-switching generally — which is a separate concept than disabling reporting — set
chi_endemic == chi_epidemic and mu_j_epidemic_factor == 0; see
Regime switching for the full inert configuration.)
delta_reporting_cases does double duty¶
It is worth being explicit about this because it surprises people. delta_reporting_cases is
read twice every tick:
- At
infectious.py:196-201, to fetch the lagged symptomatic prevalencemodel.people.Isym[t - delta_reporting_cases]for the epidemic-regime flag that controls the disease-mortality multiplier \(\mu_{jt}\). - At
infectious.py:260-268, to fetch the laggednew_symptomatic[t - delta_reporting_cases]that gets thinned by \(\rho\) forreported_cases.
So changing delta_reporting_cases shifts both the regime probe and the reporting lag together.
This is intentional: both effects are about "what surveillance saw recently," and modelling them
on separate clocks would create awkward edge cases where a patch is in epidemic mortality regime
but its reports are still pre-outbreak. Be aware that you cannot pull the two apart with
parameters alone.
Observed vs. estimated: the scoring game¶
The likelihood machinery is a game between two arrays of the same shape:
- Observed:
params.reported_casesandparams.reported_deaths, shape(npatches, n_obs_timesteps). These come from real surveillance data and are NaN-tolerant — missing weeks are dropped from the per-location likelihood, not zero-imputed. - Estimated:
results.reported_casesandresults.reported_deaths, written by the reporting transform above on each simulated tick.
The Analyzer component invokes
calc_model_likelihood on the final
tick, comparing the first min(obs.shape[1], estimate.shape[0] - 1) columns of each. The result
is stashed on model.log_likelihood.
A few important consequences flow from this design:
- Without observed data, the likelihood is undefined. If
params.reported_casesis missing orparams.calc_likelihoodisFalse,model.log_likelihoodis set tonan. If the likelihood call raisesValueError(mismatched shapes, negative weights, etc.) it is set to-infso calibration loops can detect and skip the failed evaluation without try/except at every call site. See the warning admonition in How to: calibrate and score. - The score is computed once. Per-tick likelihoods are not tracked. If you want a likelihood
trajectory over time, you have to compute it externally from
results.reported_cases. - Estimated counts are stochastic. Two runs of the same parameter set with different seeds will produce different scores. Calibration that ignores this Monte-Carlo noise — e.g. by running each candidate parameter set only once — will be noisy on tight likelihood landscapes.
The four-term likelihood ensemble¶
When params.calc_likelihood = True, the per-location, per-outcome Negative Binomial
log-likelihood is always computed. It is the core of the score. On top of it, four optional
shape terms can be enabled by giving them positive weights; each contributes additively. All
default weights are zero, so a fresh parameter set scores on the NB core alone.
Term 1: the NB core¶
For each location \(j\) and outcome \(\{\mathrm{cases}, \mathrm{deaths}\}\), the model evaluates a weighted Negative Binomial log-PMF:
The dispersion \(k\) is estimated from the observed data by weighted method-of-moments
(nb_size_from_obs_weighted), with a floor nb_k_min_cases / nb_k_min_deaths (default 3)
and a ceiling at \(10^5\). Because \(k\) depends only on the observations, two competing parameter
sets are scored against the same noise model — fitting "to the noise" is not a degree of
freedom.
When the estimated variance does not exceed the mean (a Poisson or sub-Poisson regime),
nb_size_from_obs_weighted returns inf and the NB log-PMF collapses to the Poisson log-PMF.
This is the safest fallback: it scores the location, but stops pretending overdispersion exists
where the data does not show any.
Term 2: peak timing¶
If weight_peak_timing > 0, for each known peak in epidemic_peaks whose peak_date falls
inside [date_start, date_stop], the model locates the estimated peak inside a \(\pm 14\)-step
window around the observed peak and scores the timing offset (converted to weeks) with a
Normal(0, sigma_peak_time) log-PDF. sigma_peak_time defaults to 1 week.
This term penalises a model that gets the right magnitude but in the wrong calendar quarter — something the NB core does only weakly, because a one-bin shift in the peak hits a small handful of timesteps but does not flatten the whole likelihood profile.
Term 3: peak magnitude¶
If weight_peak_magnitude > 0, the model takes the log-ratio of the estimated to the observed
peak height inside the same \(\pm 14\)-step window and scores it with a Normal(0, \(\tilde\sigma\))
log-PDF on the log scale. The standard deviation is adaptive:
Large observed peaks tighten the prior; small observed peaks (under 100) leave \(\tilde\sigma\) at
its base value sigma_peak_log (default 0.5). The intuition: when the observed peak is in the
hundreds or thousands of cases, the relative noise is small and the model should be held to a
tight magnitude match; when the observed peak is a handful of cases, the relative noise is
enormous and the term should not dominate the score.
Term 4: cumulative progression¶
If weight_cumulative_total > 0, the model evaluates the NB log-PMF of the cumulative obs vs.
estimated counts at a sequence of fractional timepoints, by default at 25%, 50%, 75%, and 100%
of the series length. The dispersion is scaled by the number of summed timesteps
(\(k \cdot \mathrm{end\_idx}\)), and the contribution from each timepoint is normalised by
\(\mathrm{end\_idx}\) before averaging — this makes the cumulative term roughly per-observation
and comparable in scale to the NB core.
This term penalises models that get the per-tick shape mostly right but accumulate a systematic under- or over-count over the season.
Term 5: WIS¶
If weight_wis > 0, the model computes the Weighted Interval Score at a set of quantile
levels (default \(\{0.025, 0.25, 0.5, 0.75, 0.975\}\)). The WIS uses the NB (or Poisson, when \(k\)
collapses) quantile function evaluated at each timestep to build symmetric prediction intervals,
then scores each observation against those intervals with an interval-coverage penalty plus a
median absolute-error term. Lower WIS is better, so the term is negated before being added to
the score.
T-normalisation: putting the four terms on a common scale¶
The shape terms are internally T-normalised so that the user-supplied weights live on a common scale:
The ratio \(N_{\mathrm{obs}} / N_{\mathrm{component}}\) scales each shape term so that with a
weight of 1.0, the term contributes roughly as much signal as the NB core. A weight of 0.25
therefore means about a quarter as much signal as the core. This makes the weights interpretable
without knowing the exact length of the observation series in advance.
The location-level total \(\ell_j\) is multiplied by weights_location[j] and summed over all
locations to produce the scalar score.
How epidemic_peaks flows through the system¶
The peak-timing and peak-magnitude terms both need a list of known epidemic peaks per
location. That list lives in params.epidemic_peaks, a pandas DataFrame with columns iso_code
and peak_date. Its trip through the system has three stages worth knowing about:
- Ingestion.
dict_to_propertysetex(the helper that buildsparamsfrom the raw config) converts theepidemic_peaksfield to a DataFrame and auto-adds aloc_idxcolumn that maps eachiso_codeto the integer row inparams.location_name. So by the timecalc_model_likelihoodsees the DataFrame, every peak knows which simulation row it belongs to. - In-window filter. Inside
calc_model_likelihood, any peak whosepeak_datefalls outside[date_start, date_stop]is silently dropped. This matters because the naivenp.argmin(|date_seq - peak_date|)would otherwise snap an out-of-window peak to eithert=0ort=n_time_steps-1, biasing the shape terms. This was a post-f8e3b38correction that brought the Python implementation into line with the upstream MOSAIC R code. - Bypass. If neither
weight_peak_timingnorweight_peak_magnitudeis positive, theepidemic_peaksDataFrame is never read; the field can be omitted from the parameter set entirely. The validator only checks the column schema when the field is present.
If epidemic_peaks is present but date_start / date_stop are missing — or if the daily /
weekly date sequence doesn't match the simulation length — _peak_idx_lists stays None and
both peak terms silently contribute zero. The simulation still runs and the score is still
produced; you just don't get any peak-shape signal.
How it connects to the rest of the model¶
The reporting transform lives inside the Infectious
component, which fires once per tick during the main loop. By the end of the run,
results.reported_cases and results.reported_deaths are fully populated.
On the final tick, the Analyzer component fires and
invokes calc_model_likelihood, pulling all available optional arguments out of params (any of
weight_cases, weight_deaths, weights_time, weights_location, nb_k_min_cases,
nb_k_min_deaths, the four weight_* shape weights, sigma_peak_time, sigma_peak_log,
epidemic_peaks, date_start, date_stop). The scalar result is written to
model.log_likelihood. From there it is yours to consume — calibration sweeps typically pickle
the model, read log_likelihood, and discard the rest.
The reporting machinery is downstream of the dynamics but tightly coupled to one piece of them:
the regime-switching factor \(\chi^{\mathrm{eff}}\) shares the epidemic threshold with the
disease-mortality multiplier \(\mu_{jt}\). If you want to reason about either in isolation, see
Regime switching — and remember that the
canonical inert values for the lagged regime probe are chi_endemic == chi_epidemic and
mu_j_epidemic_factor == 0, not zero \(\chi\) values (which would cause a divide-by-zero in the
reporting transform).
See also¶
- Reporting parameter reference — exhaustive per-field
list of
rho,rho_deaths,delta_reporting_cases,delta_reporting_deaths,calc_likelihood, the weight / sigma scoring fields, andepidemic_peaks. - Regime switching parameter reference —
chi_endemic,chi_epidemic,epidemic_threshold,mu_j_epidemic_factor, and the inert configurations for both reporting inflation and epidemic-mortality regimes. - How to: calibrate and score — operational recipe for
setting
calc_likelihood = True, supplying observed data, choosing shape-term weights, and reading the scalar score offmodel.log_likelihood. - Model overview — where the reporting transform sits in the per-tick pipeline and how it relates to the SEIRV state.