Skip to content

Optimize PatternPairAggregator with incremental parsing#3607

Open
KYash03 wants to merge 3 commits intopipecat-ai:mainfrom
KYash03:optimize-pattern-aggregator
Open

Optimize PatternPairAggregator with incremental parsing#3607
KYash03 wants to merge 3 commits intopipecat-ai:mainfrom
KYash03:optimize-pattern-aggregator

Conversation

@KYash03
Copy link

@KYash03 KYash03 commented Jan 31, 2026

Replaced the regex loop in PatternPairAggregator with a single-pass parser. It’s significantly faster and consumes less memory. Verified backward compatibility with existing tests!

@markbackman
Copy link
Contributor

This change is pretty significant. Is there any way to make it more narrowly focused on this?

  • Improved PatternPairAggregator performance by using incremental delimiter detection.

@KYash03
Copy link
Author

KYash03 commented Feb 1, 2026

I've trimmed the diff to remove unrelated changes. The core change is replacing regex-based re.finditer scanning (O(n²)) with incremental delimiter detection (O(n)) — that's inherently a full rewrite of the matching logic, so the diff can't get much smaller without artificially splitting it into separate PRs. Happy to do that if you'd prefer.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants