Decision
What action is someone trying to take?
Cut, keep or reallocate marketing spend?Foundations · Module 01
Think Like an Analyst.
The first skill is not SQL. It is turning a vague business concern into a question that data can actually answer.
Core idea
A stakeholder usually gives you a story. Your job is to turn that story into a testable analytical problem.
01
Context
Monday morning. Orders are down.
“Completed orders are down around 12% this week. I think paid marketing stopped working. Can you tell me which campaigns we should cut before tomorrow?”
The tempting response is to open SQL, compare campaigns and look for a bad channel. That would be fast — and possibly completely wrong.
The stakeholder has already supplied a cause: marketing stopped working. Treat that as a hypothesis, not a fact.
02
Concept
Start with the decision.
Metric
What exactly changed?
Completed orders, by a clearly defined completion date.Drivers
What mechanisms could move that metric?
Demand, conversion, completion, frequency, mix, measurement.Evidence
What would support or reject each explanation?
Trends, cuts, funnel rates and operational signals. Rule
Do not ask “what query should I write?” until you can explain what decision the query is supposed to improve.
03
Driver tree
Break one metric into possible causes.
Demand
Are fewer people arriving or intending to order?
Conversion
Are users failing before checkout or order creation?
Completion
Are more placed orders being cancelled or failing?
Frequency
Are existing customers ordering less often?
Mix
Did city, platform, channel or customer mix change?
Measurement
Did tracking, definitions or data pipelines change?
This is not about creating the perfect framework. It is about preventing tunnel vision. If you only investigate marketing, you can only discover marketing-shaped explanations.
04
Challenge
Frame the analysis before seeing the answer.
Exercise · 10–15 minutes
Write the analytical brief.
- What decision is the Head of Growth trying to make?
- How would you define “completed orders” so the metric is unambiguous?
- List at least four competing hypotheses for the decline.
- What data would you inspect first — before campaign-level performance?
- Which cuts would you use: city, platform, channel, new vs returning, or something else?
- What result would make you recommend not cutting marketing?
Reveal the first metric snapshot
| Metric | Previous week | Current week | Change |
|---|---|---|---|
| Sessions | 500,000 | 516,000 | +3.2% |
| New users | 82,000 | 86,000 | +4.9% |
| Checkout starts | 92,000 | 89,000 | -3.3% |
| Completed orders | 72,000 | 63,400 | -11.9% |
| Cancellation rate | 8.2% | 15.1% | +6.9 pp |
| Average order value | $24.80 | $25.10 | +1.2% |
| Paid marketing spend | $180k | $188k | +4.4% |
Fictional teaching data. The goal is to reason from the pattern, not memorize the numbers.
Now update your hypotheses.
Traffic and new users increased. Paid spend increased too. Completed orders still fell sharply, while cancellation rate nearly doubled.
That does not prove marketing is healthy. But it should move marketing lower on your initial list and move order completion / cancellation much higher.
05
AI assist
Use AI to challenge your plan — not replace it.
Analytical brief prompt
You are reviewing an analysis plan for an on-demand grocery business.
Business question:
Completed orders fell about 12% week over week. The Head of Growth believes paid marketing is the cause and is considering cutting campaigns.
My current hypotheses:
1. Demand / acquisition changed
2. Checkout conversion changed
3. Order cancellations increased
4. Existing customer frequency declined
5. City or platform mix changed
6. A tracking or metric-definition issue exists
Your job:
- Challenge this hypothesis set.
- Identify missing explanations.
- Ask for definitions you would need before trusting the analysis.
- Suggest the minimum first-pass metrics and cuts.
- Do not invent data.
- Do not conclude the cause. Why this works
The AI is being used as a critic of the analytical plan. You still own the business definition, the evidence and the final conclusion.
06
Validation
Before you touch SQL.
01
Did I define the decision the stakeholder needs to make?
02
Did I define the primary metric before diagnosing it?
03
Did I separate symptoms from possible causes?
04
Did I create multiple competing hypotheses?
05
Did I identify the minimum useful cuts of the data?
06
Did I write down what evidence would change my mind?
07
Did I avoid treating correlation as proof of cause?
07
Worked solution
One defensible first-pass plan.
Determine whether there is enough evidence to reduce or reallocate paid marketing spend, or whether another part of the customer/order journey better explains the order decline.
Distinct completed orders attributed to the order completion timestamp, excluding test/fraud orders and using the same late-arriving-data policy across both weeks.
Decompose the order decline through traffic → checkout starts → placed/completed orders, then inspect cancellation rate, customer frequency and mix by city/platform/new-vs-returning.
The snapshot weakens the “marketing failed” explanation because sessions and new users rose. The cancellation increase is a stronger lead, but still needs segmentation and operational validation.
Example analyst update
“Orders are down 11.9% week over week, but top-of-funnel demand does not appear to be the main driver: sessions are up 3.2% and new users are up 4.9%. The clearest first-pass anomaly is cancellation rate, which increased from 8.2% to 15.1%. I would hold off on cutting marketing until we identify where cancellations increased — by city, platform, customer type and cancellation reason — and confirm that the metric/data pipeline is consistent across both weeks.”