Every predictive model is trained on a finite set of conditions — a particular range of temperatures, patient ages, soil types, or market regimes. When the model is applied to conditions outside this training domain, it is extrapolating, and the reliability of its predictions may degrade rapidly. Extrapolation domain analysis (EDA) provides systematic methods for detecting when extrapolation is occurring and quantifying how much additional uncertainty it introduces.
From a Bayesian perspective, extrapolation is a natural consequence of the posterior predictive distribution. In regions where training data are dense, the posterior is well-informed and predictions are precise. As the prediction point moves away from the training data, the posterior widens — the model becomes less certain. Bayesian models automatically signal extrapolation through increased predictive uncertainty, provided the model is properly specified.
Decomposition Epistemic uncertainty (model uncertainty) grows with distance from training data
Aleatoric uncertainty (noise) remains approximately constant
Key Principle As x* moves outside the training domain,
epistemic uncertainty dominates → total uncertainty increases
Applicability Domain Methods
In quantitative structure-activity relationship (QSAR) modeling and cheminformatics, the concept of an applicability domain formalizes the region of chemical space where model predictions are reliable. Methods include leverage-based approaches (examining the hat matrix from training), distance-based approaches (measuring proximity to training points in feature space), and probability-based approaches (estimating the density of the training distribution at the query point).
Bayesian approaches unify these ideas: the posterior predictive distribution naturally encodes the applicability domain through its uncertainty. A Gaussian process model, for instance, reverts to the prior mean and prior variance as the prediction point moves away from training data — a built-in extrapolation warning system.
Gaussian process (GP) models are particularly well-suited for extrapolation domain analysis because their predictive uncertainty has a clear interpretation. In the training region, the GP interpolates with low uncertainty determined by the noise level. Outside the training region, the predictive variance grows toward the prior variance, signaling that the model has no data-driven information. This behavior is automatic — no separate extrapolation detection algorithm is needed. The kernel function determines how quickly uncertainty grows with distance, providing a natural length scale for the applicability domain.
Climate and Environmental Applications
Extrapolation domain analysis is critical in climate science, where models trained on historical conditions must project future conditions that may be unprecedented. A species distribution model trained on current climate data must extrapolate to predict species ranges under future warming. A crop yield model calibrated to historical weather must predict yields under novel temperature-precipitation combinations. In both cases, identifying where extrapolation occurs and how much additional uncertainty it introduces is essential for honest risk assessment.
Connection to Bayesian Uncertainty Quantification
Bayesian uncertainty quantification naturally distinguishes between interpolation (predictions within the support of the training data) and extrapolation (predictions outside it). The posterior predictive distribution provides calibrated uncertainty in both regimes — but only if the model is correctly specified. Model misspecification can produce overconfident predictions even in extrapolation, making prior predictive checks, sensitivity analysis, and model comparison essential complements to EDA.
Active learning and Bayesian experimental design leverage EDA directly: by identifying regions of high uncertainty (often at the boundary of the training domain), they guide data collection to the most informative locations, systematically expanding the domain over which the model makes reliable predictions.
"All prediction is extrapolation to some degree. The question is not whether we are extrapolating but how far, and whether our uncertainty estimates honestly reflect the distance." — William Welch and colleagues, "Screening, Predicting, and Computer Experiments" (1992)