Add semantic model parsing architecture doc (docs/arch/3.3_Semantic_Models.md)#12765
Add semantic model parsing architecture doc (docs/arch/3.3_Semantic_Models.md)#12765theyostalservice wants to merge 3 commits intomainfrom
Conversation
… AGENTS.md Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #12765 +/- ##
==========================================
+ Coverage 91.41% 91.49% +0.07%
==========================================
Files 203 203
Lines 25844 25945 +101
==========================================
+ Hits 23626 23739 +113
+ Misses 2218 2206 -12
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
QMalcolm
left a comment
QMalcolm
left a comment
There was a problem hiding this comment.
This is on the right track! Thank you for putting this together. One thing we should consider before moving forward is what is the status of v1 specification. Supported but not encouraged? That is we shouldn't break it, but it is not where improvements/changes shoudl happen.
|
|
||
| ## Overview | ||
|
|
||
| Semantic models are first-class resources in dbt-core that expose model data to MetricFlow for metric computation. They define the *entities*, *dimensions*, and *measures* of a model in terms the Semantic Layer can query. Parsing produces `SemanticModel` nodes in the manifest, which are later validated by `dbt_semantic_interfaces`. |
There was a problem hiding this comment.
which are later validated by
dbt_semantic_interfaces
Soon to be out of date 😂 No change needed here yet, just found it entertaining
| Defined as an independent entry under a top-level `semantic_models:` key in any schema YAML file: | ||
|
|
||
| ```yaml | ||
| semantic_models: | ||
| - name: revenue | ||
| model: ref('fct_revenue') | ||
| entities: | ||
| - name: transaction | ||
| type: primary | ||
| dimensions: | ||
| - name: ds | ||
| type: time | ||
| type_params: | ||
| time_granularity: day | ||
| measures: | ||
| - name: revenue | ||
| agg: sum | ||
| expr: amount | ||
| ``` | ||
|
|
||
| Parsed by `SemanticModelParser.parse()` in `schema_yaml_readers.py`. The semantic model is a fully independent entry in the YAML; its `model: ref('...')` field links it to the referenced model node via `depends_on`. |
There was a problem hiding this comment.
Is v1 deprecated? I.e. do we want to no longer encourage the authoring of v1 metrics? If so we should probably note that in this file.
There was a problem hiding this comment.
I'll update it with a note. The answer is that V2 YAML should be the default in all things going forward, but there are several specific situations where v1 supports things v2 does not, and we are not able to deprecate v1 at this time.
2 participants
Why
docs/arch/has detailed architecture docs for parsing (3_Parsing.md, 3.1_Partial_Parsing.md) but nothing covering semantic model parsing. Investigating DI-3697 required ~20 minutes of exploratory code reading to reconstruct knowledge that would have taken 2 minutes with a reference doc. Adding this now so future contributors (and AI agents) can orient quickly.What
New:
docs/arch/3.3_Semantic_Models.md— covers:SchemaSourceFiletracking fields (semantic_models, node_patches, metrics_from_measures, etc.)Updated:
AGENTS.md— adds an "Architecture Documentation" section at the top pointing todocs/arch/with a quick-reference table of key docsNew - added another doc for troubleshooting SL parsing issues. This is related to a number of user requests, but was generated out of the work done for Improve error message for unknown fields in semantic_model config #12766 .
Refs
Drafted by claude-sonnet-4-6 under the direction of @theyostalservice