Why be a Bayesian

A non-technical introduction to the Bayesian approach

Bayesian
AI life science
course material
Bayesian inference is not only elegant and intuitive, but also helpful for overcoming common challenges with data from the life sciences
Author

Daniel S. Mazhari-Jensen

Published

May 26, 2026

Slides

The slides contain speaking notes that you can view by pressing ‘S’ on the keyboard.

Why be a Bayesian?

Why do we need a Bayesian approach to regression?

Can’t we just use ordinary least squares?

The answer is not (only):

  • philosophy
  • elegance
  • a matter of preference

Bayesian regression stabilizes inference by:

  1. Explicitly stating expectations
  2. Constraining parameters, offering regularization
  3. Constraining model, offering stability in parametrization of the model

Real regression problems are often weakly identified, noisy, unstable, and geometrically pathological.

What is the alternative?

Frequentist statistics is by definition neither worse nor better. But the common statistical practice in todays academic publications is often lacking the robustness and explicit interpretation needed to drive home a correct sound and trustworthy interpretation of the estimand.

Is there a fight?

No! Bayesian and frequentist approaches co-exists. While there has been philosophical debate, the main obsticale for bayesian statsistics was computation power.

Bayesian approaches had gain popularity in AI for techniques such as uncertainty quantification, bayesian optimization, active learning & robotics, time series forecasting, and bayesian deep learning.

When to use Frequentist and when to use Bayesian approaches?

The Frequentist approach is often sufficient — and indeed comparable to Bayesian statsistics — when the model is:

  • simple
  • easy to parametrize
  • (multivariate) gaussian residuals
  • flat prior
  • large sample size
Code 
set.seed(18062026)

# Data
n <- 100                 # samples
x <- rnorm(n)            # x = N(0,1)      
y <- 2 + 3*x + rnorm(n)  # α = 2, β = 3, ε = N(0,1) 
# Fixed Effects

Parameter   | Coefficient |       95% CI
----------------------------------------
(Intercept) |        2.09 | [1.91, 2.28]
x           |        3.01 | [2.82, 3.20]
# Fixed Effects

Parameter   | Median |       95% CI
-----------------------------------
(Intercept) |   2.09 | [1.91, 2.27]
x           |   3.00 | [2.82, 3.20]

where the first output is frequentist maximum likelihood coefficients and confidence intervals. The second output being the median of 2000 markov chain monte carlo (MCMC) draws and the credible interval.

But sometimes, they are very different!

However, for more complex and demanding models, Bayesian approaches allows rigorous and flexible handling of uncertainty, model diagnostics, and parametrization. While a Frequentist approach is nearly always available, it often comes with stricter assumptions and more quirks (which makes solutions less intuitive and unable to transfer to other problems).

One example is generalized linear models (logistic regression, Poisson, multinomial, etc.): GLMs never guarantee a multivariate Gaussian posterior distribution. Thus, quadratic approximation will fail. While this is no issue for MCMC, frequentist always assume this!

Other times, Bayesian models are the only choice:

Suppose we have:

  • 32 observations (humans?)
  • 30 predictors (genes?)
  • strong predictor correlation (ρ ~ .8)
  • weak signal (a.u. 2)
  • noise at \(\mathcal{N}(0,2)\)

This is extremely common in:

  • social science
  • biology
  • psychology
  • economics
  • policy research

The truth is simple: only one predictor matters:

\[ y = 2x_1 + \epsilon \]

All other predictors have zero effect and all predictors are highly correlated (ρ ~ 0.8) with noise around same magnitude as signal (Normal(0,2)).

Visualizing the model:

Is this magic? No, regularization…

This is achieved using regularizing priors.

all prior \(x_n\) is \(\mathcal{N}(0, 1.5),\) The prior intercept is \(\mathcal{N}(0, 0.25)\)

So we are vocal about our assumption and expectation that the effect of each x is +-3 95% of the time.

To learn more about how to use the prior, tag along for the following exercises.

Exercises: Bayesian updating and posterior probabilities

Goal

The goal of these exercises is to experience the core idea of Bayesian inference:

We start with prior knowledge, collect data, update our beliefs, and use the posterior distribution to answer scientific questions.

You do not need to perform a full Bayesian analysis. The purpose is to understand the logic behind Bayesian inference and how posterior samples can be used.

Exercises — Updating beliefs about an assay

You have developed a new diagnostic assay.

The assay is designed to detect a pathogen.

Each test gives either:

  • success = pathogen detected
  • failure = pathogen not detected

The unknown parameter is:

\(\theta = P(\text{successful detection})\)

Your goal is to estimate the probability that the assay works.

Exercises 1 — Define a prior belief

Before collecting data, you assume that any value between 0 and 1 is equally possible.

Which of the following prior distributions would best fit your hypothesis?

Click for the solution

The correct answer is A. This corresponds to:

\(\theta \sim Beta(1,1)\)

or a uniform distribution truncated at 0 and 1.

Exercises 2 - From prior and data to posterior

Suppose we observe 8 successes out of 10 trials.

The likelihood for the unknown probability θ is shown below together with the prior from Exercise 1.

Recall that

\(Posterior ∝ Likelihood × Prior\)

For a Beta prior and Binomial data, the posterior is ​ \(θ∼Beta(α,β),\)

\(∼Binomial(n,θ),\)

which gives

\(θ∣y∼Beta(α+y, β+n−y)\)

You are not expected to calculate the posterior. Instead, use the plots to reason about where it should be.

Questions

  • Sketch what you think the posterior distribution should look like.

(Draw it directly on the figure or sketch it on separate axes.)

Compared with the prior, should the posterior

  • shift to the left,
  • stay roughly in the middle,
  • shift to the right?

Compared with the likelihood, do you expect the posterior to be

  • more spread out,
  • about equally spread out,
  • more concentrated? Explain your reasoning in one sentence.
Click for the solution

Because the prior is uniform, it gives equal weight to all values of θ. The posterior is therefore largely determined by the likelihood.

The posterior should

  • peak near \(θ≈0.8\),
  • have a shape similar to the likelihood,
  • be slightly smoother because the prior still contributes information.

In fact,

\(θ∣y∼Beta(9,3)\)

Exercises 3 — Different priors, same data

Suppose we again observe 8 successes out of 10 trials.

Below are four different prior distributions together with the same likelihood. Each panel represents a different prior belief before the data were collected.

Recall that

\(Posterior ∝ Prior × Likelihood\)

Questions

For each prior (A–D), sketch the posterior distribution.

  • Which prior do you expect to produce a posterior that is closest to the likelihood?
  • Which prior do you expect to produce a posterior that is furthest from the likelihood?
  • Rank the four posteriors from most to least concentrated (least uncertainty).
  • Explain your reasoning in one or two sentences.
Click for the solution

For a Beta prior,

\(θ∼Beta(α,β),\)

and Binomial data,

\(y∼Binomial(n,θ),\)

the posterior is

\(θ∣y∼Beta(α+y, β+n−y)\)

With y=8 and n=10, the posteriors become

|Prior | Posterior | |A: Beta(1,1) | Beta(9,3) | |B: Beta(2,2) | Beta(10,4) | |C: Beta(8,2) | Beta(16,4) | |D: Beta(2,8) | Beta(10,10) |

The likelihood favors values around \(θ≈0.8\).

  • Prior A is uninformative, so the posterior closely follows the likelihood.
  • Prior B has only a mild preference for values near 0.5, so it slightly pulls the posterior toward the center.
  • Prior C already favors large values of θ, so the posterior remains close to 0.8 and is the most concentrated.
  • Prior D favors small values of θ, which conflicts with the data. The posterior is therefore pulled away from the likelihood toward the center.

The key lesson is that the posterior balances the information from the prior and the observed data. When the prior agrees with the data, the posterior becomes more concentrated. When the prior disagrees with the data, the posterior lies somewhere between the two.