Seems like abatch_completion is not working for Claude 3.5-sonnet-latest #8761
Unanswered
yanghoonkim
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
tsia.
I ran abatch_completion + Claude 3.5 sonnet latest and found that the it took quite a long time. Looking at the cost at realtime, it is not batching the input but iterating each.
Was I doing wrong, or it is not supported yet?
Beta Was this translation helpful? Give feedback.
All reactions