Matcher with large number of patterns #10739
-
|
Hi, I have a use case where Matcher is initialized with ~800k patterns. It takes about 2 minutes (typical AWS EC2 node) to add all these patterns. Can I somehow serialize the Matcher so that it loads these patterns faster on restart? Anything else I could try? TIA |
Beta Was this translation helpful? Give feedback.
Replies: 2 comments
-
|
This has come up before, see #10514. Can you clarify why two minutes is a problem? That doesn't sound particularly slow for that amount of data if you're only doing it once per process. Also see #8790, which is about some speed issues in the Matcher we're currently working on, though it's not initialization related. |
Beta Was this translation helpful? Give feedback.
-
|
@polm, it's not a problem, but the user experience is not the best if a user has to wait that long. I mean, not the end of the world. Just wondering if it could be somehow improved. Thanks for your response and references. |
Beta Was this translation helpful? Give feedback.
This has come up before, see #10514. Can you clarify why two minutes is a problem? That doesn't sound particularly slow for that amount of data if you're only doing it once per process.
Also see #8790, which is about some speed issues in the Matcher we're currently working on, though it's not initialization related.