Allow custom StructuredOutputConverter(s) to participate in Native Structured Output#5659
Conversation
…ructured Output Signed-off-by: Filip Hrisafov <filip.hrisafov@gmail.com>
filiphr
left a comment
There was a problem hiding this comment.
Adding some explanation for some changes that are not entirely related to the initial request. In theory this could lead to some problems outside of the proposed changes here since the contract between the context and the underlying implementations differs.
If you prefer I can split this PR into 2 to handle the contract difference.
| return ChatClientResponse.builder() | ||
| .chatResponse(chatResponse) | ||
| .context(Map.copyOf(formattedChatClientRequest.context())) | ||
| .context(new HashMap<>(formattedChatClientRequest.context())) |
There was a problem hiding this comment.
This is needed due to the fact that Map.copyOf does not allow null values, whereas the contract of the context is Map<String, @Nullable Object>.
| .prompt(augmentedPrompt) | ||
| .context(Map.copyOf(chatClientRequest.context())) | ||
| .build(); | ||
| return chatClientRequest.mutate().prompt(augmentedPrompt).build(); |
There was a problem hiding this comment.
Same reason as the change of Map.copyOf
| private Map<String, Object> context = new ConcurrentHashMap<>(); | ||
| private Map<String, Object> context = new HashMap<>(); |
There was a problem hiding this comment.
Same reason as the Map.copyOf. ConcurrentHashMap does not allow null values.
This builds a bit on top of #5412.
The purpose of this is the fact that we extensively use the structured output with our own
StructuredOutputConverter(s), which do not extend fromBeanOutputConverter. Unfortunately, we now need to do something likeinstead of just passing our output converter to the
ChatClient. I came up with the workaround because I read the code and how everything works under the hood in Spring AI.From what I could see the
BeanOutputConverteris not really needed for this to work, since anyStructuredOutputConvertershould be able to provide a JSON Schema if they want to.Would appreciated your feedback on this.