SNOW-2478173: Improve single-row transpose helper for 1-column frames#3975
Merged
sfc-gh-joshi merged 8 commits intomainfrom Oct 31, 2025
Merged
SNOW-2478173: Improve single-row transpose helper for 1-column frames#3975sfc-gh-joshi merged 8 commits intomainfrom
sfc-gh-joshi merged 8 commits intomainfrom
Conversation
sfc-gh-jkew
approved these changes
Oct 30, 2025
…ngle-transpose-speedup
…ngle-transpose-speedup
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which Jira issue is this PR addressing? Make sure that there is an accompanying issue to your PR.
Fixes SNOW-2478173
Fill out the following pre-review checklist:
Please describe how your code solves the related issue.
While testing #3973, I noticed that aggregations on single-column frames/series were producing queries with JSON serialization and unnecessary UNPIVOT operations. The QC's
transpose_single_rowhelper method is used in aggregations to skip a PIVOT operation used in the general transpose case, but for transposing a 1x1 frame, we don't even need to UNPIVOT and need only re-label the index since we already know that the column's dtype will not change.This PR adds a fast path for 1x1
transpose_single_rowoperations, which replaces JSON/UNPIVOT operations with simple projections. It produces some modest performance improvements for operations on a 2000x1 frame:DataFrame.count: 1.48s -> 1.31s (11.2% improvement)DataFrame.describe: 2.64s -> 2.36s (10.9% improvement)DataFrame.nunique: 1.25s -> 1.21s (3.4% improvement)These improvements are likely to be more noticeable on frame produced from more complex queries.
This PR also adds explicit row count caching for the general transpose case. We currently cannot directly use the
transpose_single_rowpath for thetransposeAPI itself since the helper function drops the column labels of the result.