Skip to content

[FIX] Apply windows batch size#1429

Merged
marcopeix merged 15 commits intomainfrom
fix/windows_batch_size
Jan 7, 2026
Merged

[FIX] Apply windows batch size#1429
marcopeix merged 15 commits intomainfrom
fix/windows_batch_size

Conversation

@marcopeix
Copy link
Contributor

@marcopeix marcopeix commented Jan 5, 2026

Applies windows_batch_size before permuting to avoid materializing many windows into memory and running into OOM issues.

With this change we fix issues #1357 and #1297.

This PR also replaces #1421; that PR will not be needed and this is a simpler fix.

Copy link
Contributor

@elephaint elephaint left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this works; the sampling should be after we've established the eligible windows (otherwise we are sampling highly inefficiently).

Where are the windows materialized? What line of code? That's where the issue is.

@elephaint
Copy link
Contributor

elephaint commented Jan 6, 2026

Edit: doesn't work - still same error 😞

@marcopeix marcopeix merged commit 66aeb5e into main Jan 7, 2026
22 checks passed
@marcopeix marcopeix deleted the fix/windows_batch_size branch January 7, 2026 15:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants