Use aiobotocore for the ChatBedrock async implementation #27443
fabiopicchi
announced in
Ideas
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Checked
Feature request
To avoid the creation of a ThreadPoolExecutor, we should evaluate using aiobotocore for ChatBedrock. It is reasonably stable and even added support to BedrockConverse.
Motivation
Better control over the application's lifecycle. I have a use case in which I spin up a background thread that schedules langchain tasks to its own event loop (main thread runs synchronous code) and I would like to avoid the creation of another ThreadPoolExecutor from this background thread to help debuggability and tame complexity.
Proposal (If applicable)
My proposal would be to keep the _astream and other asynchronous methods from the interface but reimplement them using aiobotocore.
Beta Was this translation helpful? Give feedback.
All reactions