feat(duckdb)!: Handle WEEK, WEEK(day), ISOWEEK transpilation for DATE_DIFF & other Date/Time functions

[fivetran-amrutabhimsenayachit]

Issue:
BigQuery's DATE_DIFF with WEEK uses Sunday-start (US convention), while ISOWEEK uses Monday-start (ISO standard). DuckDB only supports 'week' with Monday-start semantics. Direct transpilation produces incorrect results because they count different boundaries.

Before :

Bigquery:
bq --project_id fivetran-wild-west query --use_legacy_sql=false "SELECT DATE_DIFF(DATE '2017-12-18', '2017-12-17', WEEK) AS week_diff, DATE_DIFF(DATE '2017-12-18', '2017-12-17', WEEK(MONDAY)) AS week_weekday_diff, DATE_DIFF(DATE '2017-12-18', '2017-12-17', ISOWEEK) AS isoweek_diff"
+-----------+-------------------+--------------+
| week_diff | week_weekday_diff | isoweek_diff |
+-----------+-------------------+--------------+
|         0 |                 1 |            1 |
+-----------+-------------------+--------------+

Transpilation:
python3 -c "import sqlglot; print(sqlglot.transpile(\"SELECT DATE_DIFF(DATE '2017-12-18', '2017-12-17', WEEK) AS week_diff, DATE_DIFF(DATE '2017-12-18', '2017-12-17', WEEK(MONDAY)) AS week_weekday_diff, DATE_DIFF(DATE '2017-12-18', '2017-12-17', ISOWEEK) AS isoweek_diff\", read='bigquery', write='duckdb')[0])"
-->
SELECT DATE_DIFF('WEEK', CAST('2017-12-17' AS DATE), CAST('2017-12-18' AS DATE)) AS week_diff, DATE_DIFF(WEEK(MONDAY), CAST('2017-12-17' AS DATE), CAST('2017-12-18' AS DATE)) AS week_weekday_diff, DATE_DIFF('ISOWEEK', CAST('2017-12-17' AS DATE), CAST('2017-12-18' AS DATE)) AS isoweek_diff


Duckdb:
duckdb -c "SELECT DATE_DIFF('WEEK', CAST('2017-12-17' AS DATE), CAST('2017-12-18' AS DATE)) AS week_diff, DATE_DIFF(WEEK(MONDAY), CAST('2017-12-17' AS DATE), CAST('2017-12-18' AS DATE)) AS week_weekday_diff, DATE_DIFF('ISOWEEK', CAST('2017-12-17' AS DATE), CAST('2017-12-18' AS DATE)) AS isoweek_diff"
Binder Error:
Referenced column "MONDAY" not found in FROM clause!

LINE 1: ...), CAST('2017-12-18' AS DATE)) AS week_diff, DATE_DIFF(WEEK(MONDAY), CAST('2017-12-17' AS DATE), CAST('2017-12-18' AS...

After:

Bigquery:
bq --project_id fivetran-wild-west query --use_legacy_sql=false "SELECT DATE_DIFF(DATE '2017-12-18', '2017-12-17', WEEK) AS week_diff, DATE_DIFF(DATE '2017-12-18', '2017-12-17', WEEK(MONDAY)) AS week_weekday_diff, DATE_DIFF(DATE '2017-12-18', '2017-12-17', ISOWEEK) AS isoweek_diff"
+-----------+-------------------+--------------+
| week_diff | week_weekday_diff | isoweek_diff |
+-----------+-------------------+--------------+
|         0 |                 1 |            1 |
+-----------+-------------------+--------------+

Transpilation:
sqlglot % python3 -c "import sqlglot; print(sqlglot.transpile(\"SELECT DATE_DIFF(DATE '2017-12-18', '2017-12-17', WEEK) AS week_diff, DATE_DIFF(DATE '2017-12-18', '2017-12-17', WEEK(MONDAY)) AS week_weekday_diff, DATE_DIFF(DATE '2017-12-18', '2017-12-17', ISOWEEK) AS isoweek_diff\", read='bigquery', write='duckdb')[0])"
-->
SELECT DATE_DIFF('DAY', DATE_TRUNC('WEEK', CAST('2017-12-17' AS DATE) + INTERVAL '1' DAY), DATE_TRUNC('WEEK', CAST('2017-12-18' AS DATE) + INTERVAL '1' DAY)) // 7 AS week_diff, DATE_DIFF('DAY', DATE_TRUNC('WEEK', CAST('2017-12-17' AS DATE)), DATE_TRUNC('WEEK', CAST('2017-12-18' AS DATE))) // 7 AS week_weekday_diff, DATE_DIFF('DAY', DATE_TRUNC('WEEK', CAST('2017-12-17' AS DATE)), DATE_TRUNC('WEEK', CAST('2017-12-18' AS DATE))) // 7 AS isoweek_diff

DuckdB:
sqlglot % duckdb -c "SELECT DATE_DIFF('DAY', DATE_TRUNC('WEEK', CAST('2017-12-17' AS DATE) + INTERVAL '1' DAY), DATE_TRUNC('WEEK', CAST('2017-12-18' AS DATE) + INTERVAL '1' DAY)) // 7 AS week_diff, DATE_DIFF('DAY', DATE_TRUNC('WEEK', CAST('2017-12-17' AS DATE)), DATE_TRUNC('WEEK', CAST('2017-12-18' AS DATE))) // 7 AS week_weekday_diff, DATE_DIFF('DAY', DATE_TRUNC('WEEK', CAST('2017-12-17' AS DATE)), DATE_TRUNC('WEEK', CAST('2017-12-18' AS DATE))) // 7 AS isoweek_diff"
┌───────────┬───────────────────┬──────────────┐
│ week_diff │ week_weekday_diff │ isoweek_diff │
│   int64   │       int64       │    int64     │
├───────────┼───────────────────┼──────────────┤
│     0     │         1         │      1       │
└───────────┴───────────────────┴──────────────┘

Logic:

1. Apply Normalization:

WeekStart(SUNDAY) → "WEEK"
WeekStart(MONDAY) → "ISOWEEK"
WeekStart(other) → "WEEK(day)"

2. Convert canonical WeekStart to DuckDB's DATE_TRUNC approach

• Calculate the shift

Monday (day 1): shift = 1 - 1 = 0 (no shift needed)
Tuesday (day 2): shift = 1 - 2= -1 (shift backward 1 day)
Wednesday (day 3): shift = 1 - 3 = -2   (shift backward 2 days)
Thursday (day 4):  shift = 1 - 4 = -3   (shift backward 3 days)
Friday (day 5):    shift = 1 - 5 = -4   (shift backward 4 days)
Saturday (day 6): shift = -5 (shift back 5 days)
Sunday (day 7): shift = +1 (shift forward 1 day, wraps to next Monday-based week)

• For Week unit: WEEK(SATURDAY), Shift the dates

Eg: Jan 6 - 5 days = Jan 1 (Monday)
      Jan 7 - 5 days = Jan 2 (Tuesday)

• Apply DATE_TRUNC

DATE_TRUNC('week', Jan 1) = Jan 1 (Monday)
DATE_TRUNC('week', Jan 2) = Jan 1 (Monday)

• Calculate DATE_DIFF

DATE_DIFF('day', Jan 1, Jan 1) = 0 days
0 / 7 = 0 week 

Edge Cases:

1. With negative boundaries:

` bq query --nouse_legacy_sql "SELECT DATE_DIFF(DATE '2024-01-01', DATE '2024-01-15', WEEK(SUNDAY)) as Weeks"                                                       
+-------+
| Weeks |
+-------+
|    -2 |
+-------+
sqlglot % python -c "import sqlglot; print(sqlglot.transpile(\"SELECT DATE_DIFF(DATE '2024-01-01', DATE '2024-01-15', WEEK(SUNDAY)) as Weeks\", read='bigquery', write='duckdb')[0])"

SELECT DATE_DIFF('DAY', DATE_TRUNC('WEEK', CAST('2024-01-15' AS DATE) + INTERVAL '1' DAY), DATE_TRUNC('WEEK', CAST('2024-01-01' AS DATE) + INTERVAL '1' DAY)) // 7 AS Weeks

 sqlglot % duckdb -c "SELECT DATE_DIFF('DAY', DATE_TRUNC('WEEK', CAST('2024-01-15' AS DATE) + INTERVAL '1' DAY), DATE_TRUNC('WEEK', CAST('2024-01-01' AS DATE) + INTERVAL '1' DAY)) // 7 AS Weeks"
 
┌───────┐
│ Weeks │
│ int64 │
├───────┤
│  -2   │
└───────┘`

2. WEEK - Saturday to Sunday boundary (critical test for Sunday-start weeks)
   In BigQuery: Saturday -> Sunday crosses week boundary = 1 week
   Without fix: DuckDB treats as Monday-start weeks = 0 weeks (both in same week)

 bq query --use_legacy_sql=false "SELECT DATE_DIFF('2024-01-07', '2024-01-06', WEEK) AS week_diff"
+-----------+
| week_diff |
+-----------+
|         1 |
+-----------+

sqlglot % python3 -c "import sqlglot; print(sqlglot.transpile(\"SELECT DATE_DIFF('2024-01-07', '2024-01-06', WEEK) AS week_diff\", read='bigquery', write='duckdb')[0])"
-->
SELECT DATE_DIFF('WEEK', CAST('2024-01-06' AS DATE), CAST('2024-01-07' AS DATE)) AS week_diff

sqlglot % duckdb -c "SELECT DATE_DIFF('WEEK', CAST('2024-01-06' AS DATE), CAST('2024-01-07' AS DATE)) AS week_diff"
┌───────────┐
│ week_diff │
│   int64   │
├───────────┤
│     0     │
└───────────┘

With fix: DuckDB = 1 week

bq query --use_legacy_sql=false "SELECT DATE_DIFF('2024-01-07', '2024-01-06', WEEK) AS week_diff"
+-----------+
| week_diff |
+-----------+
|         1 |
+-----------+

sqlglot % python3 -c "import sqlglot; print(sqlglot.transpile(\"SELECT DATE_DIFF('2024-01-07', '2024-01-06', WEEK) AS week_diff\", read='bigquery', write='duckdb')[0])"
-->
SELECT DATE_DIFF('DAY', DATE_TRUNC('WEEK', CAST('2024-01-06' AS DATE) + INTERVAL '1' DAY), DATE_TRUNC('WEEK', CAST('2024-01-07' AS DATE) + INTERVAL '1' DAY)) // 7 AS week_diff

sqlglot % duckdb -c "SELECT DATE_DIFF('DAY', DATE_TRUNC('WEEK', CAST('2024-01-06' AS DATE) + INTERVAL '1' DAY), DATE_TRUNC('WEEK', CAST('2024-01-07' AS DATE) + INTERVAL '1' DAY)) // 7 AS week_dif"
┌──────────┐
│ week_dif │
│  int64   │
├──────────┤
│    1     │
└──────────┘

[georgesittas]

Looks good, left a few more comments.

[georgesittas]

@fivetran-amrutabhimsenayachit can you add a test that covers the plain WEEK unit, demonstrating how it fixes the issue present in the main branch right now?

So basically, we want to have a test that covers the case where today's transpilation produces different results (e.g. 1 in bigquery, 0 in duckdb or something), and use that as a reference for how this PR fixes it.

[fivetran-amrutabhimsenayachit]

@fivetran-amrutabhimsenayachit can you add a test that covers the plain WEEK unit, demonstrating how it fixes the issue present in the main branch right now?

So basically, we want to have a test that covers the case where today's transpilation produces different results (e.g. 1 in bigquery, 0 in duckdb or something), and use that as a reference for how this PR fixes it.

@georgesittas , this is a good way to demonstrate the actual difference.
I added it in the test_bq file as well as in the PR description.

[VaggelisD]

Discussed offline, I'll try to take this to the finish line. Will go ahead and close this PR while I work on it to not confuse the context.

This is a very tricky transpilation, thank you for your work thus far @fivetran-amrutabhimsenayachit!