Run the failure
Use the red CTA to create the analytical problem intentionally.
Practice Lab · 10 · Evaluate the AI
AI SQL Red Team
An AI-generated query runs and returns a polished answer. Run validation to uncover three analytical bugs.
AI reviewSQL validationMetric logicClaim audit
Case brief
Find the duplicated join, wrong denominator and unsupported causal claim before the result reaches leadership.
01
Your task
Break it. Run the checks. Explain the evidence.
Predict the result
Before validating, decide what you expect row counts or metrics to do.
Run validation
Use the green CTA to reveal the checks that catch the failure.
Explain it
Say what broke in plain business language, not only SQL language.
02
Interactive case
Run this.
RUN THIS CASE ↓
AI SQL Red Team
Find the duplicated join, wrong denominator and unsupported causal claim before the result reaches leadership.
Ready — run the case. Plausible output passes syntax but fails meaning.
AI-generated SQLruns successfully
JOIN status_events e ON o.order_id = e.order_id
WHERE e.event_time >= CURRENT_DATE - 7
COUNT(CASE WHEN e.status='cancelled' THEN 1 END) / COUNT(*)
AS cancellation_rateAI conclusion“Marketing caused cancellations to rise.”
01Join grain
Events can multiply each order.
02Denominator
Event rows are not eligible orders.
03Claim audit
No causal evidence connects marketing to cancellations.
Three failures found: duplicated grain, wrong denominator, unsupported causal language. The query running successfully is not enough.
Debrief unlocked
What should the analyst learn?
- Fluent output is not evidence of correctness.
- Validate definitions, grain and claims independently.
- AI should accelerate analysis, not bypass analytical review.
03
Transfer
Use the same checks at work.
Fluent output is not evidence of correctness.
Validate definitions, grain and claims independently.
AI should accelerate analysis, not bypass analytical review.