-
Notifications
You must be signed in to change notification settings - Fork 30
feat!: Add invokeStructuredModel method to support new Judge online evals #970
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
|
This PR is reliant on #969 being merged first. |
|
@launchdarkly/browser size report |
|
@launchdarkly/js-sdk-common size report |
|
@launchdarkly/js-client-sdk size report |
|
@launchdarkly/js-client-sdk-common size report |
| expect(mockLogger.error).toHaveBeenCalledWith( | ||
| 'LangChain structured model invocation failed:', | ||
| error, | ||
| ); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Bug: Mismatched Logging Levels for Model Invocation Errors
The invokeModel and invokeStructuredModel methods in LangChainProvider.ts log caught errors using logger.warn(), but their corresponding tests expect logger.error(). This mismatch causes test failures and suggests an incorrect severity classification for these error conditions.
Note
Adds
invokeStructuredModelwith structured output support and wrapsinvokeModelwith error handling; updates tests and aligns SDK versions.src/LangChainProvider.ts):invokeStructuredModel(messages, responseStructure)usingwithStructuredOutput(...), returningStructuredResponsewith serialized raw response and zeroed usage metrics.invokeModelandinvokeStructuredModelin try/catch to returnsuccess=falseon failures and emit warnings; maintain assistant message shape on errors.StructuredResponse.__tests__/LangChainProvider.test.ts):invokeModelerror case and forinvokeStructuredModelsuccess/error paths, including metrics and logging assertions.package.json):@launchdarkly/server-sdk-aidev/peer versions to^0.12.0.Written by Cursor Bugbot for commit 4c38dcc. This will update automatically on new commits. Configure here.