Skip to content

Conversation

@KSDaemon
Copy link
Member

@KSDaemon KSDaemon commented Mar 24, 2025

This PR fixes datetime WEEK truncing for BigQuery.

Closes #9330

Check List

  • Tests have been run in packages where changes made if available
  • Linter has been run for changed code
  • Tests for the changes have been added if not covered yet
  • Docs have been added / updated if required

@KSDaemon KSDaemon marked this pull request as ready for review March 25, 2025 20:39
@KSDaemon KSDaemon requested review from a team as code owners March 25, 2025 20:39
@codecov
Copy link

codecov bot commented Mar 25, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 83.72%. Comparing base (bf09dae) to head (9982f7f).
Report is 7 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #9380   +/-   ##
=======================================
  Coverage   83.72%   83.72%           
=======================================
  Files         229      229           
  Lines       82574    82614   +40     
=======================================
+ Hits        69138    69172   +34     
- Misses      13436    13442    +6     
Flag Coverage Δ
cubesql 83.72% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@KSDaemon KSDaemon merged commit 6c8564f into master Mar 26, 2025
73 checks passed
@KSDaemon KSDaemon deleted the fix/bigquery-datetime_trunc branch March 26, 2025 09:00
@tlangton3
Copy link

tlangton3 commented Apr 3, 2025

Hi @KSDaemon

I've noticed this has been rectified in a significant number of queries generated by Cube within our instance (Thank you - this plagued us in tableau!). However, it appears that when queries are pushed down, the 'monday' parameter for the 'week()' function isn't included. My instance is hosted on Cube Cloud, so please don't hesitate to contact me on the Cube Slack channel if you require further details. Thank you!

What is sent by tableau

DATE_TRUNC(
    'WEEK',
    CAST(
      CAST(
        "CUBE_VIEW"."route_date" AS DATE
      ) AS TIMESTAMP
    )
  ) AS "tiwk:route_date:ok"

What is generated by cube:

      DATETIME_TRUNC(
        CAST(
          CAST(
            CAST(
              timestamp(`CUBE_VIEW`.route_date) AS DATE
            ) AS TIMESTAMP
          ) AS DATETIME
        ),
        week
      ) `datetrunc_utf8__`,

Thank you!

@KSDaemon
Copy link
Member Author

KSDaemon commented Apr 3, 2025

Hi @tlangton3 ! Thank you :)
Interesting.. because I fixed the templates used by pushdown flow... Thank you for posting an example of an incorrect query. I'll research what's going on!

@tlangton3
Copy link

Much appreciated @KSDaemon !

@KSDaemon
Copy link
Member Author

KSDaemon commented Apr 7, 2025

Hi @tlangton3!
I tried to reproduce the error but got no problems.

Here is my query:

SELECT DATE_TRUNC(
    'WEEK',
    CAST(
      CAST(
        orders.order_date AS DATE
      ) AS TIMESTAMP
    )
  ) AS result 
FROM
  orders
GROUP BY
  1
LIMIT
  10000;

Simply request the time dimension with date trunc.

And here is the generated SQL:

  SELECT `orders`.`datetrunc_utf8__` `result`
FROM (
  SELECT
        DATETIME_TRUNC(CAST(CAST(CAST(DATETIME(`orders`.order_date, 'UTC') AS DATE) AS TIMESTAMP) AS DATETIME), WEEK(MONDAY)) `datetrunc_utf8__`
      FROM
        (SELECT 1 AS id, 100 AS amount, 'new' status, CAST('2024-01-01' as timestamp) order_date UNION ALL SELECT 2 AS id, 200 AS amount, 'new' status, CAST('2024-01-02' as timestamp) order_date UNION ALL SELECT 3 AS id, 300 AS amount, 'processed' status, CAST('2024-01-03' as timestamp) order_date UNION ALL SELECT 4 AS id, 500 AS amount, 'processed' status, CAST('2024-01-04' as timestamp) order_date UNION ALL SELECT 5 AS id, 600 AS amount, 'shipped' status, CAST('2024-01-05' as timestamp) order_date
  ) AS `orders`  GROUP BY 1
) AS `orders`
LIMIT 10000

As you can see - it correctly produces WEEK(MONDAY).

Could you please provide a minimal reproducible data model (ideally just one self-sufficient cube) + REST/SQL Query?

@tlangton3
Copy link

tlangton3 commented Apr 9, 2025

Hello @KSDaemon, thanks for looking into this. I've sent you a message on Slack, as I can provide a more detailed query there :)

marianore-muttdata pushed a commit to MuttData/cube that referenced this pull request Jun 17, 2025
…ube-js#9380)

* fix(schema-compiler): Fix BigQuery DATETIME_TRUNC() week processing

* add tests
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Week with sql push down in BigQuery

4 participants