⚡️ Speed up method CharacterRemover.remove_control_characters by 46%
#281
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
📄 46% (0.46x) speedup for
CharacterRemover.remove_control_charactersincode_to_optimize/remove_control_chars.py⏱️ Runtime :
574 microseconds→394 microseconds(best of85runs)📝 Explanation and details
Here’s an optimized version of your program. The main bottleneck is
re.sub, which is relatively slow for simple tasks like filtering ASCII ranges, especially in tight loops. You can greatly speed this up by usingstr.translatewith a translation table that drops the unwanted control characters. This avoids regex overhead and is much faster in practice.Why is this faster?
str.translatedoes pure C-level translation and omission in a single pass, no regex engine overhead.Guaranteed same results: Control chars
chr(0)–chr(31)andchr(127)are omitted, just as with your regex.This will significantly reduce the time per call as shown in your profile. If you want even more speed and you're always working with ASCII, you can potentially use bytes, but
str.translateis already highly efficient for this use case.✅ Correctness verification report:
🌀 Generated Regression Tests Details
To edit these changes
git checkout codeflash/optimize-CharacterRemover.remove_control_characters-mbh6107band push.