duckdb1.4.2dev23: unexpected column order in pandas dataframe

[gishor]

What happens?

While testing 1.4.2dev23 we noticed that the column order in the pandas dataframe is different to the one specified in the SELECT statement. This only happens with TIMESTAMPTZ and I think it's due to the changes in src/duckdb_py/pyresult.cpp in PR #116. Would it make sense to preserve the specified order?

To Reproduce

import duckdb
to_execute = """
CREATE OR REPLACE TABLE t1 AS (
    SELECT NULL AS col1,
    NULL::TIMESTAMPTZ AS timepoint,
);

SELECT timepoint, col1 FROM t1;
"""
duckdb.sql(to_execute)
### As expected:
### ┌──────────────────────────┬───────┐
### │        timepoint         │ col1  │
### │ timestamp with time zone │ int32 │
### ├──────────────────────────┼───────┤
### │ NULL                     │  NULL │
### └──────────────────────────┴───────┘
duckdb.execute(to_execute).fetchdf()
### Column order changed:
###    col1 timepoint
### 0  <NA>       NaT

OS:

Mac

DuckDB Package Version:

1.4.2dev23

Python Version:

3.12

Full Name:

Gishor Sivanrupan

Affiliation:

ExaForce

What is the latest build you tested with? If possible, we recommend testing with the latest nightly build.

I have tested with a nightly build

Did you include all relevant data sets for reproducing the issue?

Not applicable - the reproduction does not require a data set

Did you include all code required to reproduce the issue?

• Yes, I have

Did you include all relevant configuration to reproduce the issue?

• Yes, I have

[evertlammerts]

Thanks for the report @gishor! Indeed probably related to #116, where we drop the column and then re-insert it.