This repository was archived by the owner on Mar 29, 2023. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 18
Doubling WITH statements when doing a complex EXCEPT #129
Copy link
Copy link
Open
Labels
bugSomething isn't workingSomething isn't working
Description
For some reason ibis_bigquery doubles the with statements when executing a set of projections combined with a join and except.
As example
import ibis
import ibis_bigquery as iq
t = ibis.table([('id', 'int32'), ('athlete', 'string'), ('birthday', 'date')])
m = ibis.table([('i', 'int32'), ('medals', 'int32')])
s = t.projection(['id', 'athlete'])
j = s.join(m, s['id']==m['i']).materialize()
f = j[j['athlete']=='Usain']
e = j.difference(f).projection(['id', 'athlete', 'medals'])
print(iq.compile(e))
compiles to
WITH t0 AS (
SELECT `id`, `athlete`
FROM unbound_table_2
),
t1 AS (
SELECT *
FROM t0
INNER JOIN unbound_table_3 t4
ON `id` = t4.`i`
)
SELECT t2.`id`, t2.`athlete`, t2.`medals`
FROM (
WITH t0 AS (
SELECT `id`, `athlete`
FROM unbound_table_2
),
t1 AS (
SELECT *
FROM t0
INNER JOIN unbound_table_3 t4
ON `id` = t4.`i`
)
SELECT *
FROM t1
EXCEPT
SELECT t1.*
FROM t1
WHERE t1.`athlete` = 'Usain'
) t2
As you can see the global WITH
statement is the same as the one in the global FROM
.
Besides the point but due to #87 this will also fail because EXCEPT needs to be EXCEPT DISTINCT. Now that everything migrated to ibis 2.0. I'll take another look at doing that.
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't working