Answering natural language questions skill doesn't cause the agent to check for a more appropriate metric when the request changes #64
Description
Which skill is affected?
answering-natural-language-questions-with-dbt
What happened?
The skill triggered but didn't behave as expected
What did you expect?
When choosing the wrong metric on the first run through, it should check for a more appropriate metric instead of just manually doing maths on the original dataset.
What happened instead?
❯ graph customer count and arr increase by month
⏺ I'll use the dbt Semantic Layer to query the data and then graph it. Let me start by loading the relevant skills and checking what metrics are available.
⏺ Skill(dbt:answering-natural-language-questions-with-dbt)
⎿ Successfully loaded skill
⏺ Using the dbt Semantic Layer to answer this question. Let me check what metrics are available.
...
The chart should now be open in your browser. It shows a combo chart with:
- Bars: Customer count (new customers per month) on the right axis
- Line: ARR growth on the left axis
ARR grew steadily from ~$6K in Jan 2024 to ~$135K by late 2025, while new customer additions tapered off after mid-2024 — the later ARR growth appears driven by
expansion revenue from existing customers rather than new logos.
❯ show cumulative customer count, not net new, and add the semantic layer query at the bottom of the chart so it's replicable later if needed
⏺ Done — the chart now has a text block at the bottom showing the semantic layer query details (metrics, group by, order by) and a note that cumulative values were
derived from the net new counts
❯ can the semantic layer not do running total natively?
⏺ Good question. MetricFlow does support cumulative metric types natively. Let me check — there's an active_customers metric that might already be cumulative.
⏺ Yes — active_customers is already a cumulative metric. Those numbers are close to what I manually computed (slight differences likely due to how the metric handles the
accumulation window). Let me re-query the full dataset and update the chart.
⏺ Even better — active_customers is a proper cumulative metric that also accounts for churn (peaks at 69, not 75). Let me update the chart with the correct data.