Skip to content

SNOW-2099580: fix Snowpark does not return a dataframe if default argument for called sp not given in session.call#3365

Merged
sfc-gh-yuwang merged 6 commits intomainfrom
SNOW-2099580
May 22, 2025
Merged

SNOW-2099580: fix Snowpark does not return a dataframe if default argument for called sp not given in session.call#3365
sfc-gh-yuwang merged 6 commits intomainfrom
SNOW-2099580

Conversation

@sfc-gh-yuwang
Copy link
Collaborator

@sfc-gh-yuwang sfc-gh-yuwang commented May 14, 2025

  1. Which Jira issue is this PR addressing? Make sure that there is an accompanying issue to your PR.

    Fixes SNOW-NNNNNNN

  2. Fill out the following pre-review checklist:

    • I am adding a new automated test(s) to verify correctness of my new code
      • If this test skips Local Testing mode, I'm requesting review from @snowflakedb/local-testing
    • I am adding new logging messages
    • I am adding a new telemetry message
    • I am adding new credentials
    • I am adding a new dependency
    • If this is a new feature/behavior, I'm adding the Local Testing parity changes.
    • I acknowledge that I have ensured my changes to be thread-safe. Follow the link for more information: Thread-safe Developer Guidelines
    • If adding any arguments to public Snowpark APIs or creating new public Snowpark APIs, I acknowledge that I have ensured my changes include AST support. Follow the link for more information: AST Support Guidelines
  3. Please describe how your code solves the related issue.

    Please write a short description of how your code change solves the related issue.

@sfc-gh-yuwang sfc-gh-yuwang requested review from a team as code owners May 14, 2025 21:08
@sfc-gh-snowflakedb-snyk-sa
Copy link

sfc-gh-snowflakedb-snyk-sa commented May 14, 2025

🎉 Snyk checks have passed. No issues have been found so far.

security/snyk check is complete. No issues have been found. (View Details)

license/snyk check is complete. No issues have been found. (View Details)

Copy link
Contributor

@sfc-gh-aalam sfc-gh-aalam left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would also be nice if you could describe briefly why we cannot infer this on client -> for example using describe queries or describe procedure in the PR description

Comment on lines +4126 to +4127
is_return_table: When set to a non-null value, it signifies whether the return type of sproc_name
is a table return type. This skips infer check and returns a dataframe with appropriate sql call.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we say,

When set to True, the return value of this function returns a DataFrame object. This is useful when the given stored procedure's return type is a table.

CHANGELOG.md Outdated
- Improved the error message of the XML reader when the specified row tag is not found in the file.
- Improved query generation for `Dataframe.drop` to use `SELECT * EXCLUDE ()` to exclude the dropped columns. To enable this feature, set `session.conf.set("use_simplified_query_generation", True)`.
- Added support for `VariantType` to `StructType.from_json`
- Added support for parameter `is_return_table` in `Session.call`, which control if a dataframe is returned.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- Added support for parameter `is_return_table` in `Session.call`, which control if a dataframe is returned.
- Added support for parameter `is_return_table` in `Session.call`, which can be used to set the return type of the functions a `DataFrame` object.

*args: Any,
statement_params: Optional[Dict[str, Any]] = None,
log_on_exception: bool = False,
is_return_table: Optional[bool] = None,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do you think return_dataframe is a better kwarg here? From a users perspective, I think it makes more sense.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is better, do you think we should align it with _call() ?

log_on_exception: Log warnings if they arise when trying to determine if the stored procedure
as a table return type.
return_dataframe: When set to True, the return value of this function is a DataFrame object.
This is useful when the given stored procedure's return type is a table.
Copy link
Contributor

@sfc-gh-aling sfc-gh-aling May 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's also add comment about default None behavior

#### Improvements

- Added support for reading XML files with namespaces using `rowTag` and `stripNamespaces` options.
- Added support for parameter `is_return_table` in `Session.call`, which can be used to set the return type of the functions a `DataFrame` object.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the call functions has parameter named return_dataframe while here it's called is_return_table
which name we want to use?

Copy link
Contributor

@sfc-gh-aling sfc-gh-aling left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please address my comment before merging

@sfc-gh-yuwang sfc-gh-yuwang merged commit da047c2 into main May 22, 2025
35 of 39 checks passed
@sfc-gh-yuwang sfc-gh-yuwang deleted the SNOW-2099580 branch May 22, 2025 00:51
@github-actions github-actions bot locked and limited conversation to collaborators May 22, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants