Skip to content

[Bug] Event time filtering with --sample flag doesn't respect column-level quote settings from sources.yml #11858

@pablomc87

Description

@pablomc87

Is this a new bug in dbt-core?

  • I believe this is a new bug in dbt-core
  • I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

When using the --sample flag with dbt, the event time filtering doesn't respect the column-level quote settings defined in sources.yml. This causes SQL compilation errors when using reserved keywords as column names, even when they are explicitly marked as quote: true. The errors arrive when the column name is in camel case or with spaces.

Expected Behavior

The generated SQL should respect the quote: true setting and use the column name with quotes:

-- Expected SQL filter
where "createdAt" >= '2024-01-01' and "createdAt" < '2024-01-31'

Steps To Reproduce

sources.yml

version: 2

sources:
  - name: source_name
    tables:
      - name: table_name
        columns:
          - name: "createdAt"
            quote: true
        config:
          event_time: "createdAt"

dbt command:
dbt build --target sampling --sample '1 year'

Relevant log output

21:26:36  Failure in model model_name (models/model_name.sql)
21:26:36    Database Error in model model_name (models/model_name.sql)
  column "createdat" does not exist
  LINE 24: ... * from "testdb"."source_name"."table_name" where createdAt ...
                                                                ^
  HINT:  Perhaps you meant to reference the column "table_name.createdAt".
  compiled code at target/run/test_dbt_sampling/models/model_name.sql


Compiled code:

  create  table "testdb"."sampling"."table_na__dbt_tmp"
 
    as
  
  (

select * from (select * from "testdb"."source_name"."table_name" where createdAt >= '2024-07-24 21:26:36.233275+00:00' and createdAt < '2025-07-24 21:26:36.233275+00:00') _dbt_et_filter_subq_table_name;

Environment

- OS: MacOS
- Python: 3.11.13
I've seen the issue both on dbt-core and dbt Cloud.
dbt-core version: 1.10.4
dbt Cloud version: 2025.7.24+00c7a0e
Adapter type: PostgreSQL
Adapter Version: 1.9.1post20+f492c919d3bd415bf5065b3cd8cd1af23562feb0

Which database adapter are you using with dbt?

postgres

Additional Context

By modifying the core/dbt/context/providers.py, specifically the resolve_event_time_filter method from BaseResolver class, I managed to get the appropriate compilation.

def resolve_event_time_filter(self, target: ManifestNode) -> Optional[EventTimeFilter]:
        event_time_filter = None
        sample_mode = getattr(self.config.args, "sample", None) is not None

        # TODO The number of branches here is getting rough. We should consider ways to simplify
        # what is going on to make it easier to maintain

        # Only do event time filtering if the base node has the necessary event time configs
        if (
            isinstance(target.config, (NodeConfig, SeedConfig, SourceConfig))
            and target.config.event_time
            and isinstance(self.model, (ModelNode, SnapshotNode))
        ):
            quote_config = target.config._extra.get('quote', None)
            if quote_config:
                field_name = f'"{target.config.event_time}"'
            else:
                field_name = target.config.event_time

            # Handling of microbatch models
            if (
                isinstance(self.model, ModelNode)
                and self.model.config.materialized == "incremental"
                and self.model.config.incremental_strategy == "microbatch"
                and self.manifest.use_microbatch_batches(project_name=self.config.project_name)
                and self.model.batch is not None
            ):
                # Sample mode microbatch models
                if sample_mode:
                    start = (
                        self.config.args.sample.start
                        if self.config.args.sample.start > self.model.batch.event_time_start
                        else self.model.batch.event_time_start
                    )
                    end = (
                        self.config.args.sample.end
                        if self.config.args.sample.end < self.model.batch.event_time_end
                        else self.model.batch.event_time_end
                    )
                    event_time_filter = EventTimeFilter(
                        field_name=field_name,
                        start=start,
                        end=end,
                    )

                # Regular microbatch models
                else:
                    event_time_filter = EventTimeFilter(
                        field_name=field_name,
                        start=self.model.batch.event_time_start,
                        end=self.model.batch.event_time_end,
                    )

            # Sample mode _non_ microbatch models
            elif sample_mode:
                event_time_filter = EventTimeFilter(
                    field_name=field_name,
                    start=self.config.args.sample.start,
                    end=self.config.args.sample.end,
                )

        return event_time_filter

Note the retrieval of the quote configuration and the rough double quoting at the top of the method, which provides the appropriate compilation.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingtriage

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions