Is it possible too run a pipe line considering Rate limit of LLMs. #17697
Unanswered
Sparshsing
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
When running a pipeline involving LLMs (for ex. TitleExtractor, SummaryExtractor), is there a way that only a certain number of nodes can be processed in a minute?
We could first extract the nodes from documents and then run them in batches as per rate limit. But is there a standard functionality for this, as even node parsers could sometimes use LLMs?
Beta Was this translation helpful? Give feedback.
All reactions