← All Data Visualization with AI modules

Module 9 — Bayesian Model Criticism Defense

QuantegyAI Module 9 · Canvas Project 5 · Due the date on Canvas

Update uncertainty with evidence, visualize posterior distributions honestly, and test whether a fitted model can generate data that resemble what was actually observed.

Module 9 is a guided path: 15 short pages, one idea each, with a guided check at the end of every page. Start Module 9 with the learning plan →

What to do for Canvas Project 5

Work through Module 9 and save its required evidence and files.
Complete the associated workspaces: Module 8, Module 9. Previously earned completion remains valid; additional work is optional review.
Open Project 5, check the packet status, and download Project_5_Canvas_Packet.zip. For Canvas, upload at least one artifact from your work or a screenshot of your grade. The full ZIP is optional. A download is not a Canvas submission. Submit by the date on Canvas, only if you have not already submitted.

QuantegyAI feedback does not submit to Canvas. Module numbers, evidence identifiers, and Canvas weekly resource numbers are distinct.

Statistics focusprior, likelihood, posterior, credible intervals, posterior prediction, model criticism

R/tool focusBeta-binomial updating, simulation, posterior predictive checks, reproducible AI audit

Required evidenceDV17 Bayesian defense with a defended prior and model-revision decision

Why this follows the core: Distribution theory describes possible outcomes. Bayesian inference updates uncertainty about unknown quantities, while posterior predictive checking asks whether the model can reproduce important features of the observed data.

Student learning outcomes

By the end of Module 9, students can…

Distinguish prior, likelihood, posterior, and posterior predictive distributions, and defend a prior choice on pre-data grounds.
Update a Beta-binomial model in R and verify the conjugate posterior by hand.
Interpret a credible interval as a conditional posterior probability statement — and explain how that differs from a confidence interval.
Run a posterior predictive check tied to a scientific discrepancy and read “no conflict detected” without over-claiming.
Report prior sensitivity and identify what additional data would resolve a prior-dependent decision.

Interactive reading

Select True or False before opening each explanation.

Prior · predict before reveal

Using a prior automatically makes an analysis subjective and invalid. True or false?

False. Every analysis contains assumptions. A Bayesian prior makes one assumption explicit and testable. The defensible question is whether the prior is justified and whether the conclusion is sensitive to reasonable alternatives.

Posterior · predict before reveal

After observing data, a 95% credible interval may be read as a probability statement about the parameter. True or false?

True, within the stated model and prior: the posterior assigns 95% probability to parameter values inside that interval. That is different from the repeated-sampling interpretation of a 95% confidence interval.

Sequential updating · predict before reveal

If yesterday’s posterior becomes today’s prior, two independent data batches give the same final posterior whichever order they arrive in. True or false?

True, provided the model is unchanged and the batches are conditionally independent. The likelihood contributions multiply, so processing the same evidence in a different order gives the same posterior.

Prediction · predict before reveal

A posterior that fits the parameter well shows the model describes the data-generating process adequately. True or false?

False. Parameter uncertainty is conditional on the model. Posterior predictive checks compare replicated data from the fitted model with observed features to expose model-data mismatch.

Model criticism · predict before reveal

A posterior predictive p-value close to 0.5 shows the model is true. True or false?

False. It only says the chosen discrepancy is not unusual under the fitted model. Other discrepancies may still reveal poor fit, and no finite set of checks proves a model true.

Bayesian updating lab

Open the Bayesian updating lab · about 12 minutes

This synthetic teaching example specifies 14 successes in 20 trials; it is not a collected student dataset. Change the prior and the observed successes. Compare how much the data move a weak prior versus a concentrated prior.

The Bayesian updating lab needs JavaScript.

Credible interval reasoning

Open the credible-interval reasoning challenge · about 8 minutes

The interval reasoning lab needs JavaScript.

AI-output audit

Open the Bayesian AI-output audit · about 10 minutes

This AI audit needs JavaScript.

Posterior predictive check

Open the posterior predictive model check · about 12 minutes

This separate synthetic example uses 24 successes in 30 training trials and 16 successes in 20 holdout trials. Fit a Beta-binomial model to training data, generate replicated future samples from the posterior predictive distribution, and compare an observed holdout count with those replications.

The posterior predictive lab needs JavaScript.

R lab

Open the runnable Bayesian verification R lab · about 15 minutes

Browser R · Bayesian verification

The script defines a synthetic retake-cohort fixture with 14 successes in 20 trials. External retrieval is not applicable; preserve the script version and seed with your evidence. Verify the conjugate update and simulate a posterior predictive distribution without leaving QuantegyAI.

DV17 — Bayesian Model Criticism Defense

Open the DV17 evidence

This required DV17 defense is packaged with DV15 and DV16 from Module 8 into Project_5_Canvas_Packet.zip, due the date on Canvas. It feeds the final portfolio; the Final (the date on Canvas) is review only and introduces no new instruction.

Where do I get my data?

Use the Beta-Binomial data from the R lab: a synthetic teaching fixture with a Beta(2, 2) prior and 14 successes in 20 trials. The source is the supplied script, not institutional pass-rate records; external retrieval is not applicable. Running it prints the posterior Beta(16, 8), its mean, the 2.5th, 50th and 97.5th credible limits, and a posterior predictive tail probability. That output is your Bayesian evidence brief.

Showing prior sensitivity? Change alpha and beta at the top of the same R lab and press Run again: the posterior, the interval and the predictive check all recompute, which is the comparison the second checklist item asks for. The posterior predictive check lab shows the same idea visually.

Prefer your own dataset? You can use any built-in R dataset or add your own CSV with count data under Your data in the R lab (files up to 500 MB stay in your browser). But you don't need to — the R lab dataset is ready to go.

Module 9 evidence item: DV17

Module 9 AI-feedback workspace

Open the Module 9 AI-feedback workspace

Use the Evidence Studio for formative feedback

The Bayesian defense workspace needs JavaScript.

Adaptive reflection

What I know / where I go next

I can distinguish a prior, likelihood, posterior, and posterior predictive distribution. I can explain a credible interval without borrowing confidence-interval language. I can defend a prior and test sensitivity to plausible alternatives. I can design a posterior predictive discrepancy tied to the scientific question. I can revise a model when replicated data fail to resemble observed data.

Mastery check loading…

Module 9 record

Your saved Module 9 record needs JavaScript. You can also see it on your skills dashboard report card.

Required evidence Complete the Module 9 defense and mastery check to record DV17, then open the Project 5 Canvas Packet for DV15–DV17.