Context chunking in an Agent system #10039
Unanswered
HDB-Systems
asked this question in
Questions
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
I'm building an Agent system for generating analysis of accounts receivable. I have a database containing invoices, and for every question asked by the user, the Agent queries the relevant invoices, to analyze them and then answer the question.
I thought that, in the event of the query returning a large number of invoices, the analysis would get compromised because of the large context. Does this make sense? Based on this assumption, I created a tool that first gets the entire list of invoices, and separates it into ranges of N invoices (chunks), so that each chunk gets sent to another tool that returns the corresponding invoices. The idea is to make partial analysis of these invoices subsets, so that in the end, each analysis would be concatenated into one, final analysis.
The problem is, when I saw the logs, I realized the behavior is currently not chunk generation -> partial analysis -> chunk generation -> partial analysis, etc, but instead, all chunks are generated and concatenated to the Agent's LLM context, and only then a full analysis is made, which defeats the entire idea, since the context is still large. What would be a solution to this?
Beta Was this translation helpful? Give feedback.
All reactions