Commit 1d79086
feat(pkg-py): return native DataFrame types with generic type narrowing (#196)
* feat(pkg-py): return native DataFrame types with generic type narrowing
Changes DataSource.get_data() and .execute_query() to return native
DataFrame types (pd.DataFrame, pl.DataFrame, pl.LazyFrame) instead of
narwhals wrappers, providing better IDE support and type inference.
Key changes:
- DataFrameSource now returns polars/pandas DataFrames matching input type
- PolarsLazySource returns native pl.LazyFrame from execute_query()
- Added IntoDataFrameT TypeVar for generic type capture in QueryChat
- Updated all QueryChat subclasses with 3 overloads for type narrowing:
- pl.LazyFrame → QueryChat[pl.LazyFrame]
- IntoDataFrameT → QueryChat[IntoDataFrameT] (any DataFrame type)
- sqlalchemy.Engine → QueryChat[nw.DataFrame]
- Updated df_to_html() to convert native types back to narwhals
- Fixed tests to use native DataFrame APIs (.tolist(), .iloc[])
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* refactor: address PR feedback - remove comment, rename _backend to _df_lib
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* refactor(pkg-py): make DataSource generic with configurable return types
- Add DataSourceT TypeVar to DataSource for generic return type
- DataFrameSource uses IntoDataFrameT to preserve input DataFrame type
- PolarsLazySource returns pl.LazyFrame (test_query validates then returns lazy)
- SQLAlchemySource adds return_type param ("polars"|"pandas", default "polars")
- Add read_sql_polars/read_sql_pandas helpers in _df_compat.py
- Update tests for new return types
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* refactor(pkg-py): use overloads to narrow SQLAlchemySource return type
SQLAlchemySource now uses overloaded __init__ to narrow return type based
on the return_type parameter:
- return_type="polars" (default) -> SQLAlchemySource[pl.DataFrame]
- return_type="pandas" -> SQLAlchemySource[pd.DataFrame]
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* feat(pkg-py): add pyarrow return type support for SQLAlchemySource
SQLAlchemySource now supports return_type="pyarrow" which returns
pa.Table from queries. The three options are:
- "polars" (default) -> pl.DataFrame
- "pandas" -> pd.DataFrame
- "pyarrow" -> pa.Table
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* refactor(pkg-py): make StateDictAccessorMixin generic to remove type ignores
- StateDictAccessorMixin is now generic over the DataFrame type
- Dash and Gradio QueryChat classes use StateDictAccessorMixin[DataFrameT]
- Removed redundant df() overrides and type ignore comments
- The mixin's df() method now properly returns the generic type
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* refactor(pkg-py): add proper generic typing to mod_server and remove type ignores
- mod_server now returns ServerValues[_T] where _T is the data source's type
- Removed type: ignore[arg-type] from ServerValues return
- Removed type: ignore[misc] from super().__init__() calls in _shiny.py and _streamlit.py
- Removed type: ignore[return-value] from mod_server calls and df() method
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* refactor(pkg-py): remove unused duckdb_result_to_nw function
The function is no longer used since DataFrameSource now uses
duckdb_result_to_polars and duckdb_result_to_pandas directly.
Updated tests to use duckdb_result_to_polars.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* refactor(pkg-py): inline duckdb result conversion calls
Remove duckdb_result_to_polars and duckdb_result_to_pandas functions
as thin abstractions that don't add value. Inline the .pl() and .df()
calls directly in PolarsLazySource.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* refactor(pkg-py): use IntoLazyFrameT for lazy frame type narrowing
Replace hardcoded pl.LazyFrame with IntoLazyFrameT TypeVar in overloads
to make lazy frame handling more generic and consistent with the
IntoDataFrameT pattern.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* refactor(pkg-py): clean up DataFrameSource and support pyarrow
- Remove DataOrLazyFrame legacy alias
- Remove quotes from TypeVar bounds (not needed in TYPE_CHECKING block)
- Remove inline comments from _df_lib field
- Add _convert_result helper that supports polars, pandas, and pyarrow
- Throw error for unsupported DataFrame backends
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* refactor(pkg-py): simplify SQLAlchemySource to return nw.DataFrame
Remove return_type parameter from SQLAlchemySource and just return
nw.DataFrame. The read_sql function tries polars first, then falls
back to pandas based on available libraries.
- Remove overloads and return_type parameter
- Use DataSource[nw.DataFrame] instead of DataSource[DataFrameT]
- Remove read_sql_polars, read_sql_pandas, read_sql_pyarrow functions
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix(pkg-py): fix CI failures for native DataFrame types
- Update tests to expect native DataFrames (pl.DataFrame) instead of nw.DataFrame
- Remove unsupported "Type Parameters" docstring section for quartodoc compatibility
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix(pkg-py): resolve pyright type errors for generic DataFrames
- Remove stale DataOrLazyFrame import from _querychat_core.py
- Add type: ignore comments for super().__init__ calls with generic type params
- Add type: ignore for _convert_result return statements in DataFrameSource
- Add type: ignore for attribute access on generic DataFrame types
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix(pkg-py): remove to_native() calls on native DataFrames
DataFrames returned by df() are now native types (polars, pandas), not
narwhals wrappers, so to_native() is no longer needed and would fail.
- Update 07-gradio-custom-app.py example
- Update _gradio.py update_displays function
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* refactor(pkg-py): remove type ignores and use narwhals TypeVars
- Remove all type ignores added on this branch by:
- Removing overloads from QueryChatBase (subclasses have their own)
- Using cast() and nw.from_native() for DataFrame operations
- Bounding TypeVars to IntoFrame for better type inference
- Use narwhals TypeVars directly instead of custom definitions:
- IntoFrameT (replaces DataFrameT) - bound to IntoFrame
- IntoDataFrameT - bound to IntoDataFrame
- IntoLazyFrameT - bound to IntoLazyFrame
- Standardize internal DataFrame operations with nw.from_native()
for uniform access to .shape, .to_pandas(), .to_native(), etc.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix(pkg-py): address PR feedback on type annotations
- Remove DataSourceT TypeVar, use IntoFrameT directly
- Simplify IntoDataFrame | IntoLazyFrame to IntoFrame in signatures
- Remove implementation note from QueryChatBase docstring
- Change get_current_data() return type from Any to IntoFrame
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* chore(pkg-py): remove redundant comments
Remove comments that simply describe what the code does when the code
is already self-explanatory (e.g., "Wrap in narwhals for uniform
DataFrame operations" before nw.from_native() calls).
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix(pkg-py): fix CI formatting issues
- Move IntoFrame import to TYPE_CHECKING block
- Add __all__ to _datasource.py for explicit re-exports
- Fix import sorting (ruff auto-fix)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* refactor(pkg-py): import TypeVars directly from narwhals
Instead of re-exporting IntoFrameT, IntoDataFrameT, IntoLazyFrameT
from _datasource.py, have each module import directly from
narwhals.stable.v1.typing.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* chore(pkg-py): remove unnecessary __all__ from _datasource.py
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Update changelog
* test(pkg-py): add tests for pyarrow Table support
Add TestDataFrameSourceWithPyArrow test class to verify that
DataFrameSource correctly handles pyarrow.Table inputs and returns
native pyarrow.Table results.
Also add pyarrow to dev dependencies and simplify test imports by
removing conditional checks (polars and pyarrow are now guaranteed
dev dependencies).
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix(pkg-py): fix import sorting in tests
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>1 parent f01820f commit 1d79086
File tree
18 files changed
+482
-243
lines changed- pkg-py
- examples
- src/querychat
- tests
18 files changed
+482
-243
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
9 | 9 | | |
10 | 10 | | |
11 | 11 | | |
12 | | - | |
13 | | - | |
14 | 12 | | |
15 | 13 | | |
16 | 14 | | |
17 | 15 | | |
18 | 16 | | |
19 | | - | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
20 | 30 | | |
21 | 31 | | |
22 | 32 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
47 | 47 | | |
48 | 48 | | |
49 | 49 | | |
50 | | - | |
51 | | - | |
| 50 | + | |
52 | 51 | | |
53 | 52 | | |
54 | 53 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2 | 2 | | |
3 | 3 | | |
4 | 4 | | |
5 | | - | |
| 5 | + | |
6 | 6 | | |
7 | 7 | | |
8 | 8 | | |
| 9 | + | |
9 | 10 | | |
10 | 11 | | |
11 | 12 | | |
| |||
31 | 32 | | |
32 | 33 | | |
33 | 34 | | |
34 | | - | |
| 35 | + | |
35 | 36 | | |
36 | 37 | | |
37 | 38 | | |
| |||
86 | 87 | | |
87 | 88 | | |
88 | 89 | | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
89 | 138 | | |
90 | 139 | | |
91 | 140 | | |
| |||
374 | 423 | | |
375 | 424 | | |
376 | 425 | | |
377 | | - | |
378 | | - | |
| 426 | + | |
379 | 427 | | |
380 | 428 | | |
381 | 429 | | |
| |||
408 | 456 | | |
409 | 457 | | |
410 | 458 | | |
411 | | - | |
412 | | - | |
| 459 | + | |
413 | 460 | | |
414 | 461 | | |
415 | 462 | | |
| |||
0 commit comments