Skip to main content

emulators.bayes_linear

emulators.bayes_linear

Bayes Linear emulator inspired by the hmer R package.

Uses a regression trend (OLS) plus a correlated residual process with squared-exponential kernel. Hyperparameters (correlation lengths) are estimated by maximizing a concentrated log-likelihood on the residuals.

Pure numpy/scipy – no TensorFlow or GPflow dependency.

Classes

Name Description
BayesLinear Bayes Linear emulator with OLS trend and squared-exponential residual correlation.

BayesLinear

emulators.bayes_linear.BayesLinear(
    x=None,
    y=None,
    test_fraction=0.25,
    nugget='mle',
    nugget_bounds=(1e-06, 1.0),
    ftol=1e-06,
    gtol=0.0001,
    starts=DEFAULT_STARTS,
    warm_nugget=DEFAULT_WARM_NUGGET,
)

Bayes Linear emulator with OLS trend and squared-exponential residual correlation.

This is the default emulator. It pairs an ordinary-least-squares regression trend with a correlated residual process (squared-exponential kernel), giving near-GPR-quality predictions and calibrated uncertainty at a fraction of the cost, using only numpy/scipy (no TensorFlow/GPflow).

Parameters

Name Type Description Default
x Optional[pd.DataFrame] Input data. Pandas DataFrame with columns representing parameter values. None
y Optional[pd.DataFrame] Output data. Pandas DataFrame with columns representing observations and rows representing samples. Each row must match the corresponding row in x. None
test_fraction Fraction of x and y held out for testing (scalar between 0 and 1). 0.25
nugget Variance of the noise in a single simulator run, as a fraction of the residual process variance (sigma_sq in get_hyperparameters()). It is added to the diagonal of the training correlation matrix, and sigma_sq * nugget is added to every predictive variance, so implausibility already counts run-to-run noise through the emulator: give the targets their data uncertainty only. The default, 'mle', learns one value, shared across the input space, by maximum likelihood, within nugget_bounds; for a deterministic simulator it goes to the lower bound. A number fixes the nugget, for example 1e-6, a small jitter (the default before version 2.2.0), when the simulator is known to be deterministic. 'mle'
nugget_bounds Lower and upper bounds on the nugget in the searches that learn it. The default lower bound, 1e-6, keeps a jitter that keeps the correlation matrix well conditioned. With a fixed nugget, those searches only supply starting correlation lengths (see starts). (1e-06, 1.0)
ftol Function-value tolerance for the L-BFGS-B correlation-length optimization (convergence criterion). It is relative, so it means the same at any number of training points. 1e-06
gtol Gradient tolerance for the same optimization. The search minimizes NLL / n (n training points) with an analytic gradient, so gtol applies to the gradient of NLL / n. 0.0001
starts Starting points for the hyperparameter search. The likelihood often has more than one optimum, so each start runs its own L-BFGS-B search over the correlation lengths and the nugget, within nugget_bounds. Each entry is one of: - 'warm': first fit the correlation lengths from 1 with the nugget fixed at warm_nugget (the warm-up), then search from that solution and warm_nugget. - 'midpoint': correlation lengths 1 and the nugget at the geometric middle of nugget_bounds. - A number: correlation lengths 1 and that starting nugget, clipped to nugget_bounds. With nugget='mle', the lowest NLL among those solutions, the warm-up included, wins. The warm-up is a candidate, so the learned fit’s NLL is never above that of the theta = 1 search at warm_nugget. With a fixed nugget, the same searches run, and then the correlation lengths are refitted with the nugget held at its value: from each of those solutions and from correlation lengths 1. The lowest NLL among the refits wins, so it is never above the search from correlation lengths 1 alone. A search with the nugget free can move around barriers between optima in the correlation lengths, which a search over the correlation lengths alone cannot, but no one start wins everywhere. The default is ('warm', 'midpoint', 1e-2). Every run’s NLL, nugget, iterations and message is in get_hyperparameters()['starts'], the candidates marked 'candidate' and the chosen one 'selected'. DEFAULT_STARTS
warm_nugget Nugget for the 'warm' start’s warm-up, clipped to nugget_bounds. The default, 1e-2, is run-to-run noise of 1% of the residual process variance: inside the default bounds, large enough to keep the correlation matrix well conditioned, and small enough not to hide signal. DEFAULT_WARM_NUGGET

Example

import numpy as np, pandas as pd
from historymatching.emulators.bayes_linear import BayesLinear
x = pd.DataFrame({'beta': np.random.rand(50), 'gamma': np.random.rand(50)})
y = pd.DataFrame({'peak': 3 * x['beta'] + x['gamma']})
em = BayesLinear(x, y)
em.train()
pred = em.predict(x)
pred.get_mean()  # predictive means; pred.get_variance() for uncertainty

Methods

Name Description
get_hyperparameters Return Bayes Linear hyperparameters as a JSON-serializable dict.
predict Predict using the trained Bayes Linear emulator.
predict_bounds Cheap bounds on the mean and observation variance predict returns.
print_emulator_description Display Bayes Linear emulator specifications.
train Train the Bayes Linear emulator.
get_hyperparameters
emulators.bayes_linear.BayesLinear.get_hyperparameters()

Return Bayes Linear hyperparameters as a JSON-serializable dict.

predict
emulators.bayes_linear.BayesLinear.predict(x)

Predict using the trained Bayes Linear emulator.

predict_bounds
emulators.bayes_linear.BayesLinear.predict_bounds(x)

Cheap bounds on the mean and observation variance predict returns.

The variance needs a triangular solve against all n_train training points, about n_train / d times the work of the mean. Here it is replaced by bounds, so historymatching.nroy_sampling.nroy_mask can decide most points without calling predict:

  • Mean: computed as predict does. BLAS rounding for one row can depend on the other rows in the batch, so mean_tol bounds how far it can move between calls: the error bound on the correlations (see _sq_exp_corr), each relative to itself, summed against the weights, plus rounding in the products, with a safety factor of 4.
  • Variance, lower bound: sigma2 * nugget, since the predictive variance is never negative.
  • Variance, upper bound: the variance given only the first m training points. Conditioning on fewer points never reduces the variance more. With the Cholesky factor of the training correlations, that variance reduction is the partial sum over the first m rows of the solve predict does, so it costs about (m / n_train)^2 of the full variance. m is SCREENING_PREFIX (500), or a quarter of the training points if that is fewer. The bound is widened for the correlation error and for rounding in the solve (_screening_slack), and is never above sigma2 * (1 + nugget), the variance far from all training points.

All bounds are on the output scale.

Parameters
Name Type Description Default
x pd.DataFrame Input data, with the training parameter columns. required
Returns
Name Type Description
PredictionBounds A PredictionBounds with arrays of shape (len(x),).
print_emulator_description
emulators.bayes_linear.BayesLinear.print_emulator_description()

Display Bayes Linear emulator specifications.

train
emulators.bayes_linear.BayesLinear.train()

Train the Bayes Linear emulator.

Steps: 1. Normalize inputs to [0, 1], standardize outputs. 2. OLS regression for the trend. 3. Optimize correlation lengths theta via concentrated log-likelihood. 4. Pre-compute Cholesky factor and weight vector for fast prediction.