Skip to content

Commit e6583e4

Browse files
SNOW-2213899: Enable hybrid execution by default (#3718)
Co-authored-by: Doris Lee <[email protected]>
1 parent 54cad0e commit e6583e4

File tree

3 files changed

+19
-6
lines changed

3 files changed

+19
-6
lines changed

CHANGELOG.md

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -27,12 +27,13 @@
2727
#### New Features
2828

2929
#### Improvements
30+
31+
- Hybrid execution mode is now enabled by default. Certain operations on smaller data will now automatically execute in native pandas in-memory. Use `from modin.config import AutoSwitchBackend; AutoSwitchBackend.disable()` to turn this off and force all execution to occur in Snowflake.
3032
- Downgraded to level `logging.DEBUG - 1` the log message saying that the
3133
Snowpark `DataFrame` reference of an internal `DataFrameReference` object
3234
has changed.
33-
3435
- Eliminate duplicate parameter check queries for casing status when retrieving the session.
35-
- Retrieve dataframe row counts through object metadata to avoid a COUNT(*) query (performance)
36+
- Retrieve dataframe row counts through object metadata to avoid a COUNT(\*) query (performance)
3637

3738
#### Dependency Updates
3839

src/snowflake/snowpark/modin/plugin/__init__.py

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@
55
import inspect
66
import sys
77
from typing import Union, Callable, Any
8+
import warnings
89

910
from packaging import version
1011

@@ -183,8 +184,19 @@
183184
)
184185
from modin.config import AutoSwitchBackend # isort: skip # noqa: E402
185186

187+
HYBRID_WARNING = (
188+
"Snowpark pandas now runs with hybrid execution enabled by default, and will perform certain operations "
189+
+ "on smaller data using local, in-memory pandas. To disable this behavior and force all computations to occur in "
190+
+ "Snowflake, run this line:\nfrom modin.config import AutoSwitchBackend; AutoSwitchBackend.disable()"
191+
)
192+
193+
warnings.filterwarnings("once", message=HYBRID_WARNING)
194+
186195
if AutoSwitchBackend.get_value_source() is ValueSource.DEFAULT:
187-
AutoSwitchBackend.disable()
196+
AutoSwitchBackend.enable()
197+
198+
if AutoSwitchBackend.get():
199+
warnings.warn(HYBRID_WARNING, stacklevel=1)
188200

189201
# Hybrid Mode Registration
190202
# In hybrid execution mode, the client will automatically switch backends when a

tests/integ/modin/hybrid/auto_switch_backend/test_plugin_default.py

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -3,11 +3,11 @@
33
#
44

55

6-
def test_importing_plugin_always_turns_off_auto_switch_backend():
7-
"""Test that importing snowflake.snowpark.modin.plugin always turns AutoSwitchBackend off."""
6+
def test_importing_plugin_always_turns_on_auto_switch_backend():
7+
"""Test that importing snowflake.snowpark.modin.plugin always turns AutoSwitchBackend."""
88

99
from modin.config import AutoSwitchBackend
1010

1111
import snowflake.snowpark.modin.plugin # noqa: F401
1212

13-
assert AutoSwitchBackend.get() is False
13+
assert AutoSwitchBackend.get() is True

0 commit comments

Comments
 (0)