Bayesian programming offers a disciplined framework for constructing systems that reason under uncertainty. Rather than treating probability as an add-on to conventional software, it elevates probabilistic specification to the primary organizing principle. A Bayesian program is a formal object consisting of a description (what we know) and a question (what we want to infer), providing a complete and auditable blueprint for any system that must act in the face of incomplete information.
Structure of a Bayesian Program
A Bayesian program consists of two main components: a description and a question. The description encodes all knowledge — prior distributions, structural assumptions, and conditional dependencies — while the question specifies the inference task to be performed at runtime.
The description itself has three parts:
Variables: the set of relevant quantities, both observed and latent, that define the problem space. These might include sensor readings, hidden states, actions, and outcomes.
Decomposition: a factorization of the joint distribution over all variables, expressing the conditional independence structure. This corresponds to specifying a directed graphical model, where each variable's distribution is conditioned only on its parents.
Parametric forms: the specific distributional families and parameters assigned to each conditional distribution in the decomposition. These can be learned from data, set by expert knowledge, or derived from physical models.
From Specification to Inference
Once the joint distribution is specified, any question — any conditional probability involving any subset of variables — can in principle be answered by marginalization and conditioning. The question takes the form P(Searched | Known), where Searched is the set of variables we want to infer and Known is the set of variables we have observed.
While the terms are sometimes used interchangeably, Bayesian programming as formalized by Pierre Bessière and colleagues emphasizes the complete specification methodology — variables, decomposition, and parametric forms — as a systematic engineering discipline. Probabilistic programming languages like Stan, PyMC, and Church provide computational tools for implementing such specifications but do not prescribe a particular methodology for structuring the model. Bayesian programming can be seen as the engineering methodology that probabilistic programming languages implement.
Applications in Robotics
Bayesian programming has found its most extensive applications in robotics, where agents must perceive, decide, and act under pervasive uncertainty. Pierre Bessière and colleagues at INRIA and the CNRS developed the formalism through a series of robotic applications in the 1990s and 2000s:
Robot localization was formulated as a Bayesian program where the robot's position is the searched variable, sensor readings and motor commands are known variables, and the decomposition captures the dynamics and observation models. This approach unified several previously ad-hoc localization methods under a common framework.
Sensor fusion combined information from cameras, laser range finders, and sonar arrays through a single joint distribution, with the decomposition expressing how each sensor depends on the underlying world state. The Bayesian program made explicit every assumption about sensor noise and correlation.
Human-robot interaction was modeled by including variables for human intentions and gestures, allowing the robot to infer the person's goals from observed behavior and respond appropriately.
The Methodology as Engineering Discipline
One of the distinctive contributions of Bayesian programming is its insistence on completeness of specification. Every assumption must be made explicit — there are no hidden defaults or implicit choices. This transparency has several virtues:
Auditability: every aspect of the system's behavior can be traced to a specific modeling choice. When the system makes an error, the offending assumption can be identified and revised.
Compositionality: Bayesian programs can be combined by merging their variable sets and decompositions, enabling modular development of complex systems.
Comparison: different models for the same task can be compared systematically by examining their structural assumptions and evaluating their predictive performance via marginal likelihood.
Theoretical Foundations
Bayesian programming rests on the interpretation of probability as an extension of logic — a position articulated by Cox, Jaynes, and de Finetti. Under this view, probability is not merely a model of random phenomena but a calculus of plausible reasoning applicable to any situation involving uncertainty. The two rules of probability (sum rule and product rule) are the only inference mechanisms needed; everything else follows from the specification.
"A Bayesian program is a complete and self-contained description of how an artificial agent understands and interacts with its world. It makes every assumption explicit and every inference principled."— Pierre Bessière
Connections to Broader Frameworks
Bayesian programming sits at the intersection of several intellectual traditions. It inherits the subjective probability interpretation from de Finetti and Savage, the maximum entropy reasoning of Jaynes, the graphical model formalism of Pearl and Lauritzen, and the decision-theoretic framework of Wald and Berger. By packaging these ideas into a concrete engineering methodology, Bayesian programming has made sophisticated probabilistic reasoning accessible to practitioners building real-world systems in domains ranging from autonomous vehicles to cognitive science models of human perception.