⚡️ Speed up function mask_tokens_evenly
by 2,936%
#2
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
📄 2,936% (29.36x) speedup for
mask_tokens_evenly
inblanc/utils.py
⏱️ Runtime :
89.6 milliseconds
→2.95 milliseconds
(best of300
runs)📝 Explanation and details
The optimization achieves a 2936% speedup by eliminating redundant computation in the inner loops through strategic precomputation and algorithmic improvements.
Key optimizations:
Precomputation of expensive operations: The original code called
is_token_large_enough()
for every token in every modulus iteration, resulting in 345,648 function calls. The optimized version precomputes alarge_enough_flags
list once upfront, reducing function calls to just 5,930 - a 98% reduction.Eliminated redundant
next_token
lookups: The original repeatedly computednext_token = '' if idx + 1 == len(tokens) else tokens[idx + 1]
for each modulus. The optimization precomputes anext_tokens
list once, avoiding 345,648 conditional checks.Batch masking with efficient indexing: Instead of checking masking conditions for every token in every modulus, the optimized version precomputes all mask indices per modulus using list comprehensions and explicit range logic, then applies masks in a single pass.
Direct list copying: Replaced
masked_input.append()
calls withtokens.copy()
and direct index assignment, reducing list operations from ~345K appends to simple copies plus targeted assignments.Performance characteristics by test case:
The optimization is most effective for large token sequences where the cost of precomputation is amortized across many modulus iterations, making it ideal for production NLP workloads with substantial text inputs.
✅ Correctness verification report:
🌀 Generated Regression Tests and Runtime
To edit these changes
git checkout codeflash/optimize-mask_tokens_evenly-mh2kd0zc
and push.