Skip to content

Commit 751e50c

Browse files
committed
Merge branch 'main' of github.com:snowflakedb/snowpark-python into jkew/apply.axis.1.row.index.0
2 parents ab4a197 + 975da52 commit 751e50c

File tree

156 files changed

+2554
-879
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

156 files changed

+2554
-879
lines changed

CHANGELOG.md

Lines changed: 37 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,42 @@
11
# Release History
22

3-
## 1.41.0 (YYYY-MM-DD)
3+
## 1.42.0 (YYYY-MM-DD)
4+
5+
### Snowpark Python API Updates
6+
7+
#### New Features
8+
9+
- Added support for `Session.client_telemetry`.
10+
- Added support for `Session.udf_profiler`.
11+
12+
#### Improvements
13+
14+
- Catalog API now uses SQL commands instead of SnowAPI calls. This new implementation is more reliable now.
15+
16+
#### Dependency Updates
17+
18+
- Catalog API no longer uses types declared in `snowflake.core` and therefore this dependency was removed.
19+
20+
### Snowpark pandas API Updates
21+
22+
#### New Features
23+
24+
- Added support for `Dataframe.groupby.rolling()`.
25+
- Added support for mapping `np.percentile` with DataFrame and Series inputs to `Series.quantile`.
26+
- Added support for setting the `random_state` parameter to an integer when calling `DataFrame.sample` or `Series.sample`.
27+
28+
#### Bug Fixes
29+
30+
- Fixed a bug in `DataFrameGroupBy.agg` where func is a list of tuples used to set the names of the output columns.
31+
32+
#### Improvements
33+
34+
- Add support for the following in faster pandas:
35+
- `groupby.nunique`
36+
- `groupby.size`
37+
- `concat`
38+
39+
## 1.41.0 (2025-10-23)
440

541
### Snowpark Python API Updates
642

@@ -67,7 +103,6 @@
67103
- Fixed a bug that `DataFrameReader.dbapi` (PuPr) is not compatible with oracledb 3.4.0.
68104
- Fixed a bug where `modin` would unintentionally be imported during session initialization in some scenarios.
69105
- Fixed a bug where `session.udf|udtf|udaf|sproc.register` failed when an extra session argument was passed. These methods do not expect a session argument; please remove it if provided.
70-
- Fixed a bug in `DataFrameGroupBuy.agg` where func is a list of tuples used to set the names of the output columns.
71106

72107
#### Improvements
73108

@@ -84,7 +119,6 @@
84119
- Added support for the `dtypes` parameter of `pd.get_dummies`
85120
- Added support for `nunique` in `df.pivot_table`, `df.agg` and other places where aggregate functions can be used.
86121
- Added support for `DataFrame.interpolate` and `Series.interpolate` with the "linear", "ffill"/"pad", and "backfill"/bfill" methods. These use the SQL `INTERPOLATE_LINEAR`, `INTERPOLATE_FFILL`, and `INTERPOLATE_BFILL` functions (PuPr).
87-
- Added support for `Dataframe.groupby.rolling()`.
88122

89123
#### Improvements
90124

@@ -95,16 +129,6 @@
95129
- `skew()` with `axis=1` or `numeric_only=False` parameters
96130
- `round()` with `decimals` parameter as a Series
97131
- `corr()` with `method!=pearson` parameter
98-
- `shift()` with `suffix` or non-integer `periods` parameters
99-
- `sort_index()` with `axis=1` or `key` parameters
100-
- `sort_values()` with `axis=1`
101-
- `melt()` with `col_level` parameter
102-
- `apply()` with `result_type` parameter for DataFrame
103-
- `pivot_table()` with `sort=True`, non-string `index` list, non-string `columns` list, non-string `values` list, or `aggfunc` dict with non-string values
104-
- `fillna()` with `downcast` parameter or using `limit` together with `value`
105-
- `dropna()` with `axis=1`
106-
107-
108132
- Set `cte_optimization_enabled` to True for all Snowpark pandas sessions.
109133
- Add support for the following in faster pandas:
110134
- `isin`
@@ -164,9 +188,6 @@
164188
- `groupby.median`
165189
- `groupby.std`
166190
- `groupby.var`
167-
- `groupby.nunique`
168-
- `groupby.size`
169-
- `groupby.apply`
170191
- `drop_duplicates`
171192
- Reuse row count from the relaxed query compiler in `get_axis_len`.
172193

docs/source/modin/numpy.rst

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,12 @@ NumPy ufuncs called with Snowpark pandas arguments will ignore kwargs.
3737
| ``np.full_like`` | Mapped to pd.DataFrame(value, index=range(height), |
3838
| | columns=range(width)) |
3939
+-----------------------------+----------------------------------------------------+
40+
| ``np.percentile`` | Mapped to Series.quantile, will stack a DataFrame |
41+
| | to convert to Series. Always returns an ndarray or |
42+
| | scalar like np.percentile. Does not implement any |
43+
| | arguments other than the input array and |
44+
| | percentage(s). |
45+
+-----------------------------+----------------------------------------------------+
4046
| ``np.may_share_memory`` | Returns False |
4147
+-----------------------------+----------------------------------------------------+
4248
| ``np.abs`` | Mapped to df.abs() |

docs/source/modin/supported/dataframe_supported.rst

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -384,8 +384,13 @@ Methods
384384
+-----------------------------+---------------------------------+----------------------------------+----------------------------------------------------+
385385
| ``rtruediv`` | P | ``level`` | |
386386
+-----------------------------+---------------------------------+----------------------------------+----------------------------------------------------+
387-
| ``sample`` | P | | ``N`` if ``weights`` or ``random_state`` is |
388-
| | | | specified when ``axis = 0`` |
387+
| ``sample`` | P | | ``N`` if ``weights`` is specified when |
388+
| | | | ``axis = 0``, or if ``random_state`` is not |
389+
| | | | either an integer or ``None``. Setting |
390+
| | | | ``random_state`` to a value other than ``None`` |
391+
| | | | may slow down this method because the ``sample`` |
392+
| | | | implementation will use a sort instead of the |
393+
| | | | Snowflake warehouse's built-in SAMPLE construct. |
389394
+-----------------------------+---------------------------------+----------------------------------+----------------------------------------------------+
390395
| ``select_dtypes`` | Y | | |
391396
+-----------------------------+---------------------------------+----------------------------------+----------------------------------------------------+

docs/source/modin/supported/series_supported.rst

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -383,8 +383,13 @@ Methods
383383
+-----------------------------+---------------------------------+----------------------------------+----------------------------------------------------+
384384
| ``rtruediv`` | P | ``level`` | See ``truediv`` |
385385
+-----------------------------+---------------------------------+----------------------------------+----------------------------------------------------+
386-
| ``sample`` | P | | ``N`` if ``weights`` or ``random_state`` is |
387-
| | | | specified when ``axis = 0`` |
386+
| ``sample`` | P | | ``N`` if ``weights`` is specified when |
387+
| | | | ``axis = 0``, or if ``random_state`` is not |
388+
| | | | either an integer or ``None``. Setting |
389+
| | | | ``random_state`` to a value other than ``None`` |
390+
| | | | may slow down this method because the ``sample`` |
391+
| | | | implementation will use a sort instead of the |
392+
| | | | Snowflake warehouse's built-in SAMPLE construct. |
388393
+-----------------------------+---------------------------------+----------------------------------+----------------------------------------------------+
389394
| ``searchsorted`` | N | | |
390395
+-----------------------------+---------------------------------+----------------------------------+----------------------------------------------------+

recipe/meta.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
{% set name = "snowflake-snowpark-python" %}
2-
{% set version = "1.40.0" %}
2+
{% set version = "1.41.0" %}
33
{% set noarch_build = (os.environ.get('SNOWFLAKE_SNOWPARK_PYTHON_NOARCH_BUILD', 'false')) == 'true' %}
44
{% set build_number = os.environ.get('SNOWFLAKE_SNOWPARK_PYTHON_BUILD_NUMBER', 0) %}
55

setup.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -63,7 +63,6 @@
6363
"pytest-assume", # sql counter check
6464
"decorator", # sql counter check
6565
"tox", # used for setting up testing environments
66-
"snowflake.core>=1.0.0, <2", # Catalog
6766
"psutil", # testing for telemetry
6867
"lxml", # used in XML reader unit tests
6968
]
@@ -231,6 +230,7 @@ def run(self):
231230
"opentelemetry": [
232231
"opentelemetry-api>=1.0.0, <2.0.0",
233232
"opentelemetry-sdk>=1.0.0, <2.0.0",
233+
"opentelemetry-exporter-otlp>=1.0.0, <2.0.0",
234234
],
235235
},
236236
classifiers=[

src/snowflake/snowpark/__init__.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,7 @@
4040
"QueryListener",
4141
"AsyncJob",
4242
"StoredProcedureProfiler",
43+
"UDFProfiler",
4344
]
4445

4546

@@ -54,6 +55,7 @@
5455
from snowflake.snowpark.async_job import AsyncJob
5556
from snowflake.snowpark.column import CaseExpr, Column
5657
from snowflake.snowpark.stored_procedure_profiler import StoredProcedureProfiler
58+
from snowflake.snowpark.udf_profiler import UDFProfiler
5759
from snowflake.snowpark.dataframe import DataFrame
5860
from snowflake.snowpark.dataframe_ai_functions import DataFrameAIFunctions
5961
from snowflake.snowpark.dataframe_analytics_functions import DataFrameAnalyticsFunctions
Lines changed: 180 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,180 @@
1+
#
2+
# Copyright (c) 2012-2025 Snowflake Computing Inc. All rights reserved.
3+
#
4+
5+
import keyword
6+
import json
7+
8+
from collections.abc import Mapping
9+
10+
from typing import (
11+
Any,
12+
Dict,
13+
Iterable,
14+
Iterator,
15+
Tuple,
16+
Optional,
17+
Union,
18+
)
19+
20+
21+
class ImmutableAttrDict(Mapping):
22+
"""
23+
An immutable mapping whose (identifier-valid, non-keyword, non-private) keys
24+
are also available as read-only attributes. Nested mappings are recursively
25+
wrapped as ImmutableAttrDict instances.
26+
27+
By default also recursively freezes:
28+
- dict / Mapping -> ImmutableAttrDict
29+
- list / tuple -> tuple
30+
- set -> frozenset
31+
- other types -> unchanged
32+
33+
Access:
34+
d.foo (if 'foo' is a valid identifier key)
35+
d['foo'] (always works)
36+
37+
Invalid / reserved keys (not identifiers, keywords, or starting with '_')
38+
are still accessible via item lookup but not as attributes.
39+
"""
40+
41+
__slots__ = ("_data", "_frozen")
42+
43+
# ------------- Construction / conversion helpers -----------------
44+
@classmethod
45+
def _convert(cls, value: Any) -> Any:
46+
"""Recursively convert nested structures."""
47+
if isinstance(value, cls):
48+
return value
49+
if isinstance(value, Mapping):
50+
# Ensure plain dict form for predictability
51+
return cls(value)
52+
if isinstance(value, list) or isinstance(value, tuple):
53+
return tuple(cls._convert(v) for v in value)
54+
if isinstance(value, set):
55+
return frozenset(cls._convert(v) for v in value)
56+
# For other iterables you could add handling if desired.
57+
return value
58+
59+
@staticmethod
60+
def _is_exposable_attr_name(name: str) -> bool:
61+
"""Return True if name can safely be set as an attribute."""
62+
return bool(
63+
name
64+
and name[0] != "_"
65+
and name.isidentifier()
66+
and not keyword.iskeyword(name)
67+
)
68+
69+
def __init__(
70+
self,
71+
mapping: Optional[Union[Mapping[str, Any], Iterable[Tuple[str, Any]]]] = None,
72+
**kwargs: Any,
73+
) -> None:
74+
combined: Dict[str, Any] = {}
75+
if mapping is not None:
76+
if isinstance(mapping, Mapping):
77+
combined.update(mapping)
78+
else:
79+
for k, v in mapping:
80+
combined[k] = v
81+
combined.update(kwargs)
82+
83+
# Deep-convert values first
84+
converted: Dict[str, Any] = {k: self._convert(v) for k, v in combined.items()}
85+
86+
object.__setattr__(self, "_data", converted)
87+
88+
# Expose attribute names only for "safe" keys
89+
for k, v in converted.items():
90+
if self._is_exposable_attr_name(k):
91+
object.__setattr__(self, k, v)
92+
93+
object.__setattr__(self, "_frozen", True)
94+
95+
# ------------- Factory methods -----------------
96+
@classmethod
97+
def from_json(cls, j: str) -> "ImmutableAttrDict":
98+
parsed = json.loads(j)
99+
if not isinstance(parsed, Mapping):
100+
raise TypeError("JSON root must be an object to build ImmutableAttrDict")
101+
return cls(parsed)
102+
103+
# ------------- Mapping interface -----------------
104+
def __getitem__(self, key: str) -> Any:
105+
return self._data[key]
106+
107+
def __iter__(self) -> Iterator[str]:
108+
return iter(self._data)
109+
110+
def __len__(self) -> int:
111+
return len(self._data)
112+
113+
def __contains__(self, key: object) -> bool:
114+
return key in self._data
115+
116+
def get(self, key: str, default: Any = None) -> Any:
117+
return self._data.get(key, default)
118+
119+
# ------------- Attribute fallback -----------------
120+
def __getattr__(self, name: str) -> Any:
121+
data = object.__getattribute__(self, "_data")
122+
if name in data and self._is_exposable_attr_name(name):
123+
return data[name]
124+
raise AttributeError(
125+
f"{type(self).__name__!r} object has no attribute {name!r}"
126+
)
127+
128+
# ------------- Immutability enforcement -----------------
129+
def __setattr__(self, name: str, value: Any) -> None:
130+
if name.startswith("_"):
131+
object.__setattr__(self, name, value)
132+
return
133+
if object.__getattribute__(self, "_frozen"):
134+
raise AttributeError(
135+
f"{type(self).__name__} is immutable; cannot set attribute {name!r}"
136+
)
137+
object.__setattr__(self, name, value)
138+
139+
def __delattr__(self, name: str) -> None:
140+
if name.startswith("_"):
141+
object.__delattr__(self, name)
142+
return
143+
if object.__getattribute__(self, "_frozen"):
144+
raise AttributeError(
145+
f"{type(self).__name__} is immutable; cannot delete attribute {name!r}"
146+
)
147+
object.__delattr__(self, name)
148+
149+
# ------------- Utilities -----------------
150+
def to_dict(self) -> Dict[str, Any]:
151+
"""
152+
Return a shallow copy of the underlying storage.
153+
Nested ImmutableAttrDict instances are preserved (not converted).
154+
For a fully "plain" structure you can use to_plain().
155+
"""
156+
return dict(self._data)
157+
158+
def to_plain(self) -> Dict[str, Any]:
159+
"""
160+
Recursively convert back to plain Python types:
161+
ImmutableAttrDict -> dict
162+
tuple/frozenset -> list (or list for frozenset)
163+
"""
164+
165+
def unwrap(v: Any) -> Any:
166+
if isinstance(v, ImmutableAttrDict):
167+
return {k: unwrap(v._data[k]) for k in v._data}
168+
if isinstance(v, tuple):
169+
return [unwrap(x) for x in v]
170+
if isinstance(v, frozenset):
171+
return [unwrap(x) for x in v]
172+
return v
173+
174+
return {k: unwrap(v) for k, v in self._data.items()}
175+
176+
def __repr__(self) -> str:
177+
return f"{type(self).__name__}({self._data})"
178+
179+
def __reduce__(self):
180+
return (type(self), (self._data,))

0 commit comments

Comments
 (0)