- 
                Notifications
    You must be signed in to change notification settings 
- Fork 207
Support the volatility config of function nodes #1414
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
          
     Open
      
      
            QMalcolm
  wants to merge
  17
  commits into
  main
  
    
      
        
          
  
    
      Choose a base branch
      
     
    
      
        
      
      
        
          
          
        
        
          
            
              
              
              
  
           
        
        
          
            
              
              
           
        
       
     
  
        
          
            
          
            
          
        
       
    
      
from
qmalcolm--1345-begin-templating-volatility-for-scalar-functions
  
      
      
   
  
    
  
  
  
 
  
      
    base: main
Could not load branches
            
              
  
    Branch not found: {{ refName }}
  
            
                
      Loading
              
            Could not load tags
            
            
              Nothing to show
            
              
  
            
                
      Loading
              
            Are you sure you want to change the base?
            Some commits from the old base branch may be removed from the timeline,
            and old review comments may become outdated.
          
          
  
     Open
                    Changes from 16 commits
      Commits
    
    
            Show all changes
          
          
            17 commits
          
        
        Select commit
          Hold shift + click to select a range
      
      004a784
              
                Begin templating volatility in global scalar function macro
              
              
                QMalcolm 2561a20
              
                Begin specifying minimum dbt-common version in dbt-tests-adapter to e…
              
              
                QMalcolm 2e2e2ac
              
                Add basic test cases to `dbt-tests-adapter` for different volatility …
              
              
                QMalcolm f9b1ae0
              
                Test dbt-postgres support of volatility config
              
              
                QMalcolm 97f2784
              
                Update snowflake scalar function macro to support volatility
              
              
                QMalcolm fe15244
              
                Begin testing snowflake udf volatility support
              
              
                QMalcolm 4912fc0
              
                Support volatility setting in dbt-redshift sql functions
              
              
                QMalcolm 43bd82a
              
                Begin testing volatility setting of sql functions in dbt-redshift
              
              
                QMalcolm 52aeb96
              
                Add changie docs
              
              
                QMalcolm 644d71d
              
                Test that bigquery does nothing with volatility
              
              
                QMalcolm e8c1ad4
              
                Update dbt-snowflake to issue warning when encountering `stable` vola…
              
              
                QMalcolm c897e65
              
                Update dbt-bigquery to raise warning when any volatility is present f…
              
              
                QMalcolm 8869308
              
                Update test files used in bigquery volatility function tests
              
              
                QMalcolm 88bdd67
              
                Update files used for redshift function volatility tests
              
              
                QMalcolm 9125f89
              
                Ensure redshift volatility is being templated instead of hard coded
              
              
                QMalcolm 612f20c
              
                Proper checking of `none` for bigquery volatility
              
              
                QMalcolm 0e628bf
              
                Future proof volatility macros to catch any future unknown volatilities
              
              
                QMalcolm File filter
Filter by extension
Conversations
          Failed to load comments.   
        
        
          
      Loading
        
  Jump to
        
          Jump to file
        
      
      
          Failed to load files.   
        
        
          
      Loading
        
  Diff view
Diff view
There are no files selected for viewing
        
          
  
    
      
          
            6 changes: 6 additions & 0 deletions
          
          6 
        
  dbt-adapters/.changes/unreleased/Features-20251029-164230.yaml
  
  
      
      
   
        
      
      
    
  
    
      This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
      Learn more about bidirectional Unicode characters
    
  
  
    
              | Original file line number | Diff line number | Diff line change | 
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| kind: Features | ||
| body: Add default macros for interpolating the volatility of sql UDFs | ||
| time: 2025-10-29T16:42:30.989678-05:00 | ||
| custom: | ||
| Author: QMalcolm | ||
| Issue: "1345" | 
  
    
      This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
      Learn more about bidirectional Unicode characters
    
  
  
    
              
        
          
  
    
      
          
            6 changes: 6 additions & 0 deletions
          
          6 
        
  dbt-bigquery/.changes/unreleased/Under the Hood-20251029-165603.yaml
  
  
      
      
   
        
      
      
    
  
    
      This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
      Learn more about bidirectional Unicode characters
    
  
  
    
              | Original file line number | Diff line number | Diff line change | 
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| kind: Under the Hood | ||
| body: Test that bigquery does nothing with volatility | ||
| time: 2025-10-29T16:56:03.92939-05:00 | ||
| custom: | ||
| Author: QMalcolm | ||
| Issue: "1345" | 
  
    
      This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
      Learn more about bidirectional Unicode characters
    
  
  
    
              
  
    
      This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
      Learn more about bidirectional Unicode characters
    
  
  
    
              
        
          
  
    
      
          
            6 changes: 6 additions & 0 deletions
          
          6 
        
  dbt-postgres/.changes/unreleased/Features-20251029-164109.yaml
  
  
      
      
   
        
      
      
    
  
    
      This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
      Learn more about bidirectional Unicode characters
    
  
  
    
              | Original file line number | Diff line number | Diff line change | 
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| kind: Features | ||
| body: Support volatility setting for sql UDFs | ||
| time: 2025-10-29T16:41:09.298749-05:00 | ||
| custom: | ||
| Author: QMalcolm | ||
| Issue: "1345" | 
  
    
      This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
      Learn more about bidirectional Unicode characters
    
  
  
    
              | Original file line number | Diff line number | Diff line change | 
|---|---|---|
| @@ -1,5 +1,22 @@ | ||
| from dbt.tests.adapter.functions.test_udfs import UDFsBasic | ||
| from dbt.tests.adapter.functions.test_udfs import ( | ||
| UDFsBasic, | ||
| DeterministicUDF, | ||
| StableUDF, | ||
| NonDeterministicUDF, | ||
| ) | ||
|  | ||
|  | ||
| class TestPostgresUDFs(UDFsBasic): | ||
| pass | ||
|  | ||
|  | ||
| class TestPostgresDeterministicUDFs(DeterministicUDF): | ||
| pass | ||
|  | ||
|  | ||
| class TestPostgresStableUDFs(StableUDF): | ||
| pass | ||
|  | ||
|  | ||
| class TestPostgresNonDeterministicUDFs(NonDeterministicUDF): | ||
| pass | 
        
          
  
    
      
          
            6 changes: 6 additions & 0 deletions
          
          6 
        
  dbt-redshift/.changes/unreleased/Features-20251029-163948.yaml
  
  
      
      
   
        
      
      
    
  
    
      This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
      Learn more about bidirectional Unicode characters
    
  
  
    
              | Original file line number | Diff line number | Diff line change | 
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| kind: Features | ||
| body: Support volatility setting in sql UDFs | ||
| time: 2025-10-29T16:39:48.716884-05:00 | ||
| custom: | ||
| Author: QMalcolm | ||
| Issue: "1345" | 
  
    
      This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
      Learn more about bidirectional Unicode characters
    
  
  
    
              
              
  
    
      This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
      Learn more about bidirectional Unicode characters
    
  
  
    
              
        
          
  
    
      
          
            6 changes: 6 additions & 0 deletions
          
          6 
        
  dbt-snowflake/.changes/unreleased/Features-20251029-164024.yaml
  
  
      
      
   
        
      
      
    
  
    
      This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
      Learn more about bidirectional Unicode characters
    
  
  
    
              | Original file line number | Diff line number | Diff line change | 
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| kind: Features | ||
| body: Support volatility setting in sql UDFs | ||
| time: 2025-10-29T16:40:24.708639-05:00 | ||
| custom: | ||
| Author: QMalcolm | ||
| Issue: "1345" | 
  
    
      This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
      Learn more about bidirectional Unicode characters
    
  
  
    
              
              | Original file line number | Diff line number | Diff line change | ||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|  | @@ -2,6 +2,7 @@ | |||||||||||||||||||||||||||||||
| CREATE OR REPLACE FUNCTION {{ target_relation.render() }} ({{ formatted_scalar_function_args_sql()}}) | ||||||||||||||||||||||||||||||||
| RETURNS {{ model.returns.data_type }} | ||||||||||||||||||||||||||||||||
| LANGUAGE SQL | ||||||||||||||||||||||||||||||||
| {{ scalar_function_volatility_sql() }} | ||||||||||||||||||||||||||||||||
| AS | ||||||||||||||||||||||||||||||||
| {% endmacro %} | ||||||||||||||||||||||||||||||||
|  | ||||||||||||||||||||||||||||||||
|  | @@ -10,3 +11,14 @@ | |||||||||||||||||||||||||||||||
| {{ model.compiled_code }} | ||||||||||||||||||||||||||||||||
| $$ | ||||||||||||||||||||||||||||||||
| {% endmacro %} | ||||||||||||||||||||||||||||||||
|  | ||||||||||||||||||||||||||||||||
| {% macro snowflake__scalar_function_volatility_sql() %} | ||||||||||||||||||||||||||||||||
| {% if model.config.get('volatility') == 'deterministic' %} | ||||||||||||||||||||||||||||||||
| IMMUTABLE | ||||||||||||||||||||||||||||||||
| {% elif model.config.get('volatility') == 'stable' %} | ||||||||||||||||||||||||||||||||
| {% do exceptions.warn("`Stable` function volatility is not supported by Snowflake, and will be ignored") %} | ||||||||||||||||||||||||||||||||
| {% elif model.config.get('volatility') == 'non-deterministic' %} | ||||||||||||||||||||||||||||||||
| VOLATILE | ||||||||||||||||||||||||||||||||
| {% endif %} | ||||||||||||||||||||||||||||||||
|          | ||||||||||||||||||||||||||||||||
| {% if model.config.get('volatility') == 'deterministic' %} | |
| IMMUTABLE | |
| {% elif model.config.get('volatility') == 'stable' %} | |
| {% do exceptions.warn("`Stable` function volatility is not supported by Snowflake, and will be ignored") %} | |
| {% elif model.config.get('volatility') == 'non-deterministic' %} | |
| VOLATILE | |
| {% endif %} | |
| {% set volatility = model.config.get('volatility') %} | |
| {% if volatility == 'deterministic' %} | |
| IMMUTABLE | |
| {% elif model.config.get('volatility') == 'non-deterministic' %} | |
| VOLATILE | |
| {% elif volatility is not none %} | |
| {% do exceptions.warn("user passed unsupported volatility config value: " ~ volatility ~"; it will be ignored") %} | |
| {% endif %} | 
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Okay I think I've now added logic to sufficiently handle this 🙂
      
      Oops, something went wrong.
        
    
  
      
      Oops, something went wrong.
        
    
  
  Add this suggestion to a batch that can be applied as a single commit.
  This suggestion is invalid because no changes were made to the code.
  Suggestions cannot be applied while the pull request is closed.
  Suggestions cannot be applied while viewing a subset of changes.
  Only one suggestion per line can be applied in a batch.
  Add this suggestion to a batch that can be applied as a single commit.
  Applying suggestions on deleted lines is not supported.
  You must change the existing code in this line in order to create a valid suggestion.
  Outdated suggestions cannot be applied.
  This suggestion has been applied or marked resolved.
  Suggestions cannot be applied from pending reviews.
  Suggestions cannot be applied on multi-line comments.
  Suggestions cannot be applied while the pull request is queued to merge.
  Suggestion cannot be applied right now. Please check back later.
  
    
  
    
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit:
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
also do we need to do any validation/warning if they have set something other than these options?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(also in the future if we add another option do we need to remember to come back and raise a warning?)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Core only accepts an
Enumof possible volatility typesdeterministic,stable, andnon-deterministic. So in order for a 4th option to come into existence, we'd first have to extend the enum in core. Additionally, have a hard time imagining any other volatility type being invented, but perhaps I'm being unimaginative 🤷🏻 Future proofing doesn't hurt anything.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Okay I think I've now added logic to sufficiently handle this 🙂