-
Notifications
You must be signed in to change notification settings - Fork 2
SqliteQueryGenerator._gen_sql_array_sub_selection() incompatibility with recent SQLite #21
Copy link
Copy link
Open
Labels
bugSomething isn't workingSomething isn't working
Description
When running the following select query test against recent versions of SQLite
Lines 164 to 167 in af1a58c
| # selecting keys inside an array slice | |
| out = run_select_query('select=x,c[1|h,p]') | |
| assert out[0][1] is None | |
| assert out[1][1] == [32, 0] |
it fails with exception
AssertionError: assert '[32,0]' == [32, 0]The SQL used for this select query is generated here:
pysquril/pysquril/generator.py
Lines 351 to 390 in af1a58c
| def _gen_sql_array_sub_selection( | |
| self, | |
| term: SelectTerm, | |
| parsed: Union[ | |
| ArraySpecificSingle, | |
| ArraySpecificMultiple, | |
| ArrayBroadcastSingle, | |
| ArrayBroadcastMultiple, | |
| ], | |
| ) -> str: | |
| if ( | |
| isinstance(parsed, ArraySpecificSingle) | |
| or isinstance(parsed, ArraySpecificMultiple) | |
| ): | |
| fullkey = f"and fullkey = '$.{term.bare_term}[{parsed.idx}]'" | |
| vals = 'vals' | |
| else: | |
| fullkey = '' | |
| vals = 'json_group_array(vals)' | |
| temp = [] | |
| for key in parsed.sub_selections: | |
| temp.append(f"json_extract(value, '$.{key}')") | |
| sub_selections = ','.join(temp) | |
| sub_selections = f'json_array({sub_selections})' if len(temp) > 1 else f'{sub_selections}' | |
| selection = f""" | |
| (case when json_extract(data, '$.{term.bare_term}') is not null then ( | |
| select {vals} from ( | |
| select | |
| {sub_selections} as vals | |
| from ( | |
| select key, value, fullkey, path | |
| from {self.table_name}, json_tree({self.table_name}.data) | |
| where path = '$.{term.bare_term}' | |
| {fullkey} | |
| ) | |
| ) | |
| ) | |
| else null end) | |
| """ | |
| return self._maybe_apply_function(term, selection) |
Generated SQL:
select json_array(json_extract(data, '$.x'),
(case when json_extract(data, '$.c') is not null then (
select vals from (
select
json_array(json_extract(value, '$.h'),json_extract(value, '$.p')) as vals
from (
select key, value, fullkey, path
from test_table, json_tree(test_table.data)
where path = '$.c'
and fullkey = '$.c[1]'
)
)
)
else null end)
) from test_table Value we're getting back from run_select_query('select=x,c[1|h,p]'):
[[1900, None], [None, '[32,0]'], [88, None], [107, None], [10, None]]So far I've confirmed this behavior when using SQLite 3.39.5 and 3.40.1. After skimming the SQLite release notes, my hunch is that JSON behavior most likely changed in 3.38.0, where JSON support was promoted to built-in and some effort was made to align behavior with PostgreSQL/MySQL, but I have yet to confirm anything.
### Tasks
- [ ] Confirm which exact SQLite version introduces this new behavior
- [ ] Add support for new behavior without breaking backwards compatibility
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't working