Semantic layer
Metric contracts, glossary, entities and approved dimensions.
AI-Native Analytics · Module 11
Analyst Workflows & Agents.
Move from one-off prompting to analytical systems that know the metrics, use trusted tools, validate outputs and escalate uncertainty.
Core idea
A useful analytics agent is not just an LLM with database access. It is context + tools + rules + evaluation + escalation.
01
Context
Build a daily revenue analyst that people can trust.
“Why did revenue fall yesterday?” should trigger a governed workflow, not unrestricted SQL generation.
An agent needs stable definitions, schema metadata, verified examples, access controls and clear rules about what it may conclude.
The goal is not to remove analysts. It is to automate repeatable analytical steps while preserving human ownership of ambiguity and high-impact decisions.
02
Agent architecture
Build the system around trust boundaries.
Tool layer
Read-only query tools, verified templates and scoped data access.
Evaluation layer
Golden questions, control totals and regression tests.
Human layer
Escalate ambiguity, low confidence and consequential recommendations.
Rule
Automate stable questions first. Ambiguous questions should become analyst work, not agent guesses.
03
Challenge
Design the Revenue Analyst Agent.
Exercise · 15–25 minutes
Define the contract.
- Which questions may the agent answer autonomously?
- Which metric definitions must be injected every time?
- What verified SQL examples should seed the workflow?
- What data-quality checks run before explanation?
- When should the agent say “I need analyst review”?
- How will you test changes to the agent over time?
04
AI assist
Make escalation part of the workflow.
Agent instruction skeleton
You are a governed revenue-analysis agent.
You may:
- query approved read-only tables,
- use approved metric definitions,
- compare standard time periods,
- return descriptive findings.
You must:
- run freshness and reconciliation checks,
- state the metric definition used,
- cite the query/result supporting each claim,
- distinguish observation from cause.
Escalate to a human analyst when:
- metric definitions conflict,
- data quality checks fail,
- a requested dimension is undocumented,
- causal explanation is requested without experimental evidence,
- the recommendation could materially change spend or operations.05
Validation
The agent design checklist.
01
Are business definitions stored outside the free-form prompt?
02
Are data permissions scoped to the minimum required access?
03
Does the workflow run integrity checks before explanation?
04
Can the agent expose the evidence behind each claim?
05
Are escalation conditions explicit and testable?
06
Is there a regression suite for known questions?
06
Worked takeaway
Scale the workflow, not the ambiguity.
Repeatable descriptive questions with stable definitions.
Approved data, metrics, tools and output formats.
Known-answer tests and data-quality gates.
Ambiguity, causality and high-impact decisions to humans.
Analyst habit
The best agent design question is not “what can the model do?” It is “what decisions are safe to automate under explicit evidence and controls?”