Run the failure
Use the red CTA to create the analytical problem intentionally.
Practice Lab · 17 · Experiments & Causal Thinking
Experiment Integrity Check
Treatment conversion looks better, but the assignment split is wrong. Run integrity checks before reading lift.
A/B testsSample ratio mismatchGuardrailsCausality
Case brief
Decide whether the experiment result can support a causal conclusion.
01
Your task
Break it. Run the checks. Explain the evidence.
Predict the result
Before validating, decide what you expect row counts or metrics to do.
Run validation
Use the green CTA to reveal the checks that catch the failure.
Explain it
Say what broke in plain business language, not only SQL language.
02
Interactive case
Run this.
RUN THIS CASE ↓
Experiment Integrity Check
Decide whether the experiment result can support a causal conclusion.
Ready — run the case. Treatment/control counts drift far away from the planned 50/50 split.
Integrity check flags a severe allocation mismatch versus the planned 50/50 split. Investigate assignment/exposure before making a causal claim.
Debrief unlocked
What should the analyst learn?
- Experiment integrity comes before effect interpretation.
- Unexpected allocation can invalidate an otherwise exciting lift.
- Do not turn a broken experiment into a confident recommendation.
03
Transfer
Use the same checks at work.
Experiment integrity comes before effect interpretation.
Unexpected allocation can invalidate an otherwise exciting lift.
Do not turn a broken experiment into a confident recommendation.